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Chairman:     Malay  Ghosh 

Major  Department:     Statistics 

This  dissertation  is  concerned  with  sequential  estimation  of  the 
multivariate  normal  mean,  estimation  of  the  regression  coefficient  in 
a  normal  linear  regression  model,  and  estimation  of  the  difference  of 
mean  vectors  of  two  multivariate  normal  distributions  in  the  presence 
of  unknown  and  possibly  unequal   variance-covariance  matrices. 

For  estimating  the  p(_>3)   variate  normal   mean,  we   consider  two 
different  situations.     In  one  case,   the  covariance  matrix  is  known  up 
to  a  multiplicative  constant;   in  the  other  situation,   it  is  entirely 
unknown  but  diagonal.      In  both  cases,   the   sample  mean   is  the  maximum 
likelihood  estimator  of  the  population  mean.     When   the  covariance 
matrix  is  known  up  to  a  multiplicative  constant,  a   class  of  James- 
Stein  estimators  is  developed  which  dominates   the   sample  mean   under 
sequential    sampling  schemes  of  M.   Ghosh,   B.K.   Sinha,   and 
N.   Mukhopadhyay   ([1976]   Journal    of  Multivariate  Statistics  6, 
281-294).     Asymptotic  risk  expansions  of  the   sample  mean  vector  and 


VI 


James-Stein  estimators  are  provided  up  to  the   second  order   term. 
Additionally,   in  this  case,   some  Monte  Carlo  simulation  is  done  to 
compare   the  risks  of  the   sample  mean  vector,    the  James-Stein 
estimators,  and  a  rival   class  of  estimators.     In   the  second  case,  a 
class  of  James-Stein  estimators  is  given  which  dominates  the  sample 
mean  asymptotically  by  considering  second  order  risk  expansions. 

The  next  case  is  concerned  with  estimation  of  regression 
parameters  in  a  Gauss-Markoff  setup.     Here   the  classical   estimator  of 
the  regression  coefficient  is  the  least  squares  estimator,  and  the 
sampling  scheme   used  is   that  of  N.   Mukhopadhyay   ([1974]   Journal   of 
the   Indian  Statistical   Association   12,   39-43).     Once  again,  a  class 
of  James-Stein  estimators  that  dominates  the  least  squares  estimator 
is  developed,  and  asymptotic  risk  expansion  is  given  for  both  the 
least  squares  and  James-Stein  estimators. 

Finally,  we  consider  the  estimation  of  the  difference  of  two 
normal  mean  vectors,  and  the  sampling  schemes  developed  in  1984  at 
the  Institute  of  Applied  Mathematics  National  Tsing  Hua  University  by 
R.   Chou  and  W.   Hwang.     A  class  of  James-Stein  estimators   that 
dominates  the  difference  of  sample  mean  vectors  is  given.     Asymptotic 
risk  expansions  are  also  provided. 
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CHAPTER  ONE 
INTRODUCTION 


1.1     Sequential   Sampling 

There  are   two  basic  purposes  for  which  sequential  methods  are 
used  in  statistics.     The  first  is  to  reduce  the  sample  size  on  an 
average  as  compared  to  the  corresponding  fixed  sample  size  procedure 
which  meets  the  same  error  requirements.     Wald's  sequential 
probability  ratio  test  is  a  classic  example  of  this.     The  second  is 
to  solve  certain  problems  which  cannot  be  solved  by  any  fixed  sample 
procedure.     One  important  example  of  the  latter  is  the  fixed  length 
interval   estimation  of  the  normal   mean  with  unknown  variance  when  the 
confidence  coefficient  is  specified  in  advance.     Another  important 
problem  directly  related  to  the  topic  of  this  dissertation  is  a  point 
estimation  problem  which  is  described  below. 

Let  X^  X2,    ...   be  a  sequence  of  independent  and  identically 
distributed  N(9,a2)  variables  where  QeR1  is  unknown.     Based  on 
X1§   .    .    .    ,  Xn,  an  estimator  6n  =  an(Xx,...,Xn)   is  used  for 
estimating  0.     Suppose  the  loss  incurred  is  given  by 


L(0,6n)   =  A|6n  -  ep  +  en     , 


(1.1) 


where  A>0  is   the   known  weight  and  c(>0)   denotes  the  known  cost  per 
sampling  unit.     The  most  natural   estimator  of  0   from  a   sample  of  size 


1 


n  is  X     =  —  zn.    ,X.  with  associated  risk   (expected  loss) 
n       n     i=l  i 


R(9,X   )   =  EL(9,XJ   =  AE|Xn  -  e|2  +  en 
n  n  n 


=  A  i-+  en   .  (1.2) 


The  above  risk  is  minimized  with  respect  to  n  at 


2  v2 

n  »  n.  *   C|  (A-2-)   |]  , 


where  [|u|]  denotes  the  integer  closest  to  u.     Here,   for  simplicity, 

a2    V2 
we  will  assume  that  (A—)         is  an  integer.     However,   for  unknown 

a2  there  does  not  exist  any  fixed  sample  size  which  minimizes  (1.2) 

simultaneously  for  all   0  .      In   this  case,  motivated  from  the  optimal 

fixed  sample  size  n^^   (when  a     is  known),   the  following  sequential 

procedure  is  proposed  for  determining  the  sample  size. 


N  =  inf  (n  >  m  (>2):n  >   (A  s2/c)   '2  }  (1.3) 


where  m  denotes  the  initial   sample  size  and 


t-ik  ^i<xi-V2« 


(1.4) 


for  all   n   >  2. 


The  above   sequential   procedure  was  essentially  introduced  by 
Robbins   (1959)  who  considered  case  L(e»5n)   =  A | 6n  -   e|    +  en.     Later, 
Starr   (1966)   considered   the  more  general    case  L(e,6n)   = 
A |  6n  -  e|s  +  en1-  with  s>0  and  t>0. 

The  sequential   sampling  procedure  as  introduced  in   (1.3)   has 
sometimes  been  described  as  ad  hoc.     Robbins   (1959)  did  not  give  any 
decision  theoretic  motivation  for  such  a  procedure,  which  as  we  shall 

see  below  can  be  justified  from  the  minimax  criterion. 

?  2 

With  this  end,  assume  that  <r  is  known.     Suppose   0  has  a  N(y,r  ) 

prior.     Then   the  posterior  distribution  of  Q  given  X^   =  x^, 
1=1,..., n,   is 


2  2/n 

N(    u  + T—  (xn  -   u)    ,        °  ' %      )    • 

T2  +  5_  1   +  3— 

n  n   2 


Here,   under  the  loss   (1.1),   the  Bayes  estimator  of  9  is 


<■    x    0 

n 


iK  -  u) 


2     wn 

nx 


2 
with  corresponding  posterior  risk  equal  to  A  5 5—+  en, 

1  +  (uV) 

The  posterior  risk  being  independent  of  X.,  ...  ,  X  ,  the 


sequential   Bayes  rule  becomes  a   fixed  sample  rule  with  sample 

size,   nD   (determined  by  minimizing   the  posterior  risk)   and  estimator 
B 

5n    <*.). 

B  r  i 

Next,  consider  a  sequence  of  iN(u,m),  m>l}  priors.  Then  the 

associated  sequence  of  Bayes  estimators  is  v  + 5 (X  -  v) 

a2,  1  +  (<^/nm)       n 

with  posterior  risks  A 4^ +  en.     Since  the  sequence  of 

1  +  (CT  /nm)  2     *  * 

posterior  risks  converges  (as  m*00)   to  K  =  (A°  /n^  +  cn^  and 

R(9,X  )  =  K  for  all   9  then  the  fixed  sample  rule  with  sample  size 

determined  by  minimizing   (1.2)  with  respect  to  n  is  a  minimax  rule. 

In  summary,   n, ,   the  required  sample  size,  and  X  ic,   the 
1  n: 

estimator,   is  a  minimax  rule.     Therefore,  one  can  conceive  the 

stopping  rule   (1.3)  as  an  empirical  minimax  stopping  rule  and  not  as 

ad  hoc  as  one  might  think.     (By  empirical  we  mean  having  any  unknowns, 

in  this  case  <*  ,   replaced  by  their  estimators,   in  this  case  sn). 

Next,  consider  the  multivariate  analogue  to  the  above  problem, 

i.e.,   let  X, ,   L,   ...  be  a  sequence  of  independent  and  identically 

distributed  N   (9,c2Y)   random  vectors   (p  >  3),  with  9eRp  unknown  and 

V,  positive  definite,   known.     Based  on  X,,   .    .    .    ,   X^,   if 

5     s  S  (X ,X  )  is  used  to  estimate  Q,   suppose  the  loss  incurred 

~n       ~n  ~1  ~n  ~ 

is  given  by 


L(£.£n)  =  (£n  "  !)     S  (£n  -  Q)  +  en  , 


(1.5) 


where  Q  denotes  a  known  positive  definite   (weight)  matrix,  and  c(>0) 
is   the  known  cost  per   sampling  unit.     Again,    from  a   sample  of  size  n, 


the  most  natural   estimator  of  e   is  X„  = -r  eJ.iX.,  with  risk   (expected 
loss) 


R(9,a2,Xn)   =  n"1  a2  tr   (Q  V)   +  en 


(1.6) 


2 

As  before,   if  a     is  known,   the  above  risk  is  minimized  at 

*  V2 

n  =  n     =  (tr  QY/c)         (here,   too,  we  shall   assume  that  n     is  an 

integer).     Once  again,   if  o     is  unknown  there  does  not  exist  any 
fixed  sample  size  which  minimizes   (1.6)   simultaneously  for  all  o   , 
Again,  motivated  by  the  definition  of  n_,  we  propose  the  following 
sequential   sampling  scheme. 


N  =  inf  {n  >  m(>2)    :  n  >   (s2/c)   '2  (tr  Q  V)   /2 


(1.7) 


where  m  denotes  the  initial   sample  size  and 


-s^Hn-np}-1!^.  -xn)'  v"1  (x.  -xn> 


(1.8) 


for  every  n  >  2. 

As  can  easily  be  seen,   the  above  stopping  rule  is  a  multivariate 
generalization  of  the  univariate   stopping  rule   (1.3).     Ghosh  et  al. 
(1976)  and  Woodroofe   (1977)   considered  more  general    stopping  rules 
where  £  was  an  unknown  arbitrary  positive  definite  matrix.     To  show 
that  the   stopping  rule   (1.7)   is  more   than   just  an  ad  hoc  extension  of 
(1.3),  we  will   motivate   it  again   from  a  minimax  criterion. 


2  p 

Assume   a     is  known  and  suppose   8  has  a  N(u,t  B)   prior,  where  B 

is  a   positive  definite  matrix.     Then,   for   fixed  n,    the  posterior 
distribution  of  0  given  X.   =  x.,   i=l,...,n,   is 


2  2 

N(u  +  t2B(t2B  +  ~  Vf  ^x,   -  u)    ,    x2B  -  t23(t23  +  —  V)"1    x2B) 


with 


T2B-x2B(x23+f  V)-1    A.^fYtA+^V)"1    x2B} 


v       nx 


where   I     is   the  identity  matrix  of  order  p.     Since   the   loss   (1.5)    is 
quadratic,    the  3ayes  estimator  £n  of  0  is  again   the  mean  of  its 
posterior  distribution,   i.e., 


2 

&fy   =  H  +  t2B(t23  +f-V)_1(Xn   -  U) 
mi  ~ti         ~  ~      ~      n     ~        ~n       ~ 


2 

u  +   (I n  +-2-T  V   B)"1^  -   u) 
~p       ^2  ~  ~         mi        ~ 


with  the  corresponding  posterior  risk  equal  to 


2 
F"  trC&  I   (ID  +  ^-B^V)"1]  +  cn 


Again,   the  posterior  risk  is  independent  of  the  X.'s  so  that  the 
Bayes  sequential   decision  rule   is  a   fixed  sample  rule  with  sample 
size,  ng,  determined  by  minimizing  the  posterior  risk  with  respect  to 
n. 

Next,  consider  the  sequence  of  {N(u,mB),m  >  l}  priors  on  ®. 
Then   the  Bayes  estimator  of  0  becomes 


2 

6H  (U  =  y  +  (L  +^rv  B)_1(Xn  -  »»)  , 

n     ~n         ~        ~pnm~~         ~n       ~     ' 


with  associated  posterior  risk 


2  2 

—  tr[Q  V   (I     +i-B"1V)"1]  +  en 
n  ~  ~    ~p       nm  ~     ~ 


By  using  a  limiting  Bayes  argument  as  in  the  univariate  case  we  see 

2 
that  the  posterior  risk  converges   (as  m-*-00)    to  K  =  —  tr(Q  V)   +  en 

2  -  * 

where  R(6,o  ,x   )  =  K  for  all   G.     Hence,  n   ,   the  sample  size,  and  X   .. 

P 
the  estimator,   is  a  minimax  rule.     Consequently,    the  proposed 

stopping  rule   (1.7)  can  also  be  viewed  as  an  empirical   minimax 

stopping  rule. 

It  is  the  stopping  rule   (1.7)  and  its  eventual  modifications  to 

fit  a   regression  problem,   its  generalization   to  a  more  complex 

covariance   structure  and  its  adaptation   to  a   two  sample   situation, 

that  will   be  of  interest  to  us   (as  far  as  sequential    sampling  goes) 

in   the   subsequent  chapters. 


1.2     Shrinkage  Estimation 


Consider   the   situation   X. ,   X?,    .    .    .   are   independent  and 
identically  distributed  N   (0,a  Y)   random  vectors,   0eRp  is  unknown,   V 
is  a  known  positive  definite  matrix  and  p  >  3.     Again,   the  problem  is 
to  estimate  0.     For  a  sample  of  size  n,  assume   the  loss  (1.5)   in 
estimating  0  by  5     =  6n(X, ,...,X   ).     The  minimax  estimator 

of  0  is  X     =—    zn.   .X..     However,  Stein   (1955)  was  able  to  establish 
~        ~n       n       i =1  i 

2 
the  existence  of  estimators  of  0  (for  known   a     and  for  Q  =  V   =   I    ) 

~  ~      ~       ~P 

which  dominated  X     (and  hence  were  minimax  themselves).     Later  James 
~n 

and  Stein   (1961)  exhibited  some  explicit  estimators  dominating  X   . 
Since   then,    there   have  been  a  multitude  of  articles  concerning 

extensions  of  the  James-Stein  estimators   to  more  general    situations. 

2 

Specifically,  we  refer  to  Berger  (1976)  when  a  ,  Q  and  V  are  known, 

2 

Efron  and  Morris  (1976)  when  Q  =  V  =  I  and  a     is  unknown,  and 

Berger  and  Bock  (1976)  when  Q  is  a  known  diagonal  matrix  and  the 
covariance  matrix  is  diagonal  and  unknown.  For  a  general  class  of 
estimators  that  dominate  the  usual  mean  vector,  we  refer  to  Baranchik 
(1970),  Strawderman  (1971),  Alam  (1973),  Efron  and  Morris  (1976), 
Berger  (1976)  and  Berger  (1976). 

To  motivate  the  idea  of  a  shrinkage  estimator  let  us  consider  the 

2 
case  Q  -  V  -   I  .  Suppose  0  has  a  N(u,t  I  )  prior.  Then,  by  previous 

results,  the  Bayes  estimator  of  0  given  X.  =  x.,  i=l,...,n,  is 


B  -  t2 

5  (x   )   =  y  + =■  (x     -  w) 

~n  ~n        ~        „       CT2     ~n       ~ 

n 


=  „  +  (i  -       g2/n     )(xn  -  u) 
n 


Here,   if  <J*  and  tfc  are  unknown  then   they  must  be  estimated  from  the 
data.     With  this  end,  note  that  marginally 


,-        ,.2    .    a2        :• 


IX     -  yir  ~  (t-  +  — )   x     , 
~n       ~  n         p  ' 


where  B.I  denotes  the  usual  Euclidean  norm.  Hence, 


nx  -  u"2 
~n   ~   -1     l 

Tc    +  

n 


Also,  marginally 


2 
~2  a  2 

sn  ~  (n-l)p  x(n-l)p   • 


~2 
where  we  are  referring   to  definition   (1.8)   of  s   .     Consequently, 


~2  ? 


10 


Therefore,   replacing   the  unknowns  in   6   (X   )   by   their  estimates  we 
have 


JS  (p_2)sn 

""  ~  mix     -  uii2    ~"       ~ 


which  is  known  as  the  James-Stein  estimator.     The  above  empirical 
Bayes  interpretation  of  James-Steim  estimators  is  due  to  Efron  and 
Morris   (1973).      In  James  and  Stein   (1961),    the  estimator  was  studied 
from  a  purely  frequentist  perspective. 


JS 
For  better  understanding  of   the   shrinking  property  of   fi     ,   note 


that 


v  v     ,JS 

jn      z,    -    jn        z 


(p-2)s2 
if  and  only  if  iiX     -  yi  >   |1 


~n       ~    -    '  5  2    '   ~n 

niiX„   -   yii 


Therefore,   the  distance  between  X     and  y  will   be  greater  than   that 

,c                                                                (P"2)s2 
of  5      and  y  if  and  only  if  1  >   |1  -  — 11--.  |    .     When   this  is 

nnX-  -  w« 

JS  - 

true   6       will    shrink  X     towards   the  prior  mean   u.     To  assure   that 
~n  ~n  Z 

the  above  condition   is  always  met  we   sometimes  use  the  plus  rule 

version  of   5       given  by 
~n     J 


11 


JS      +                              (p_2)Sn        +     - 
(    C    )       =    V    +    (1    "   2-y)        (X       -    U)     , 

~n       ~ 


where   (a)     =  max  (0,a) . 

JS 
The  James-Stein  estimator,   5     ,  and  its  eventual  modifications 

~n 

to  fit  a  regression  problem,   its  generalizations  to  a  more  complex 
covariance  structure  and  its  adaptation   to  a   two-sample  problem  will 
be  of  interest  to  us   (as  far  as  estimation  goes)   in   the  subsequent 
chapters. 


1.3     Literature  Review 

There  are  only  two  pertinent  articles  to  date  that  address  the 
inadmissibility  of  the  sample  mean  vector,   X  ,as  an  estimator  of  0  in 
a  setup  where  the  sample  size  is  random.     Ghosh  and  Sen   (1983-84) 
consider  the  case  of  arbitrary  Q  and  V,   but  consider  a  two-stage 
sampling  scheme.     The  main  criticism  of   this  approach  is   that  a   two 
stage  procedure  requires  on  an  average  more  observations  than  n*.   the 
optimal    sample   size  when   ac  is  known   (see  for  example  Ghosh  and 
Mukhopadhyay,   1976).     Takada   (1984),  on  the  other  hand,   considered 
the  case  Q  =  V  =  I     under  a   full    sequential    sampling  scheme.     The 
main  objection   to  his  approach  is   that  at  the  n-th  stage  of  sampling 
his  estimator   (similar   to  the  James-Stein  estimator)   uses  only  p  of 
the  available   (n-l)p  degrees  of  freedom  to  estimate   o^.     This  will    be 
made  clear   in  Chapter  2. 
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Before  leaving  this  section  let  us  familiarize  ourselves  with  a 
particular  identity  from  Stein   (1981)  which  will   be  used  repeatedly 
in  all    the  proofs.     First  we  require   the   following   lemma. 

Lemma   1.1     Suppose  Y  ~  N ( 0 , 1 ). .     Then,   if  E[|g'(Y)|]   <  -,   one   has 


E[g  (Y)]  =  E[Yg(Y)]. 


(1.9) 


Proof:     Write  My)   =  — — exp  (--^-y2)  and  note  that  *  (y)  =  -y<t>(y) 


2tt 


Now, 


E[g'(Y)]  =  /      g'(y)<fr(y)dy 


(1.10) 


=  /    g' (y)*(y)dy  +  /      g'(y)*(y)dy 
0 


/     g'(y)[/     z$(z)dz]dy 
0  y 


0  y 

+  /       g'(y)[/       -Z4»(z)dz]dy 


/     [g(z)-g(0)]z<Kz)dz  +  /      [g(0)-g(z)](-z$(z))dz 
0  -0° 


=  /       [g(z)-g(0)jz$(z)dz 


ECYg(Y)]  +  g(0)E[Y] 

E[Yg(Y)]    , 
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since  E[Y]  =  0.     This  brings  us  to  the  identity,  referred  to  as 
Stein's  identity,  which  is  stated  as  follows: 


Lemma   1.2     Let  X  ~  NO, a").     Then   if  E         [|h   (X)|]   <  ~,  one   has 
e.cr 


E  ?[h'(x)]  =\e   [(x-e)h(x)]  . 

9,cT         a        9,ct 


(1.11) 


x  -  0, 


Proof:  Define  g(x)  by  g(x)  =  h(e  +  ax).  Then,  h(x)  =  g  ( — — )  and 


1     ',x 


X  -  e, 


h  (x)  =-g   (-*-^-).     Note  that  Y  =  i^-z— )  ~  N(0.1).     Hence, 


E        2[h'(x)]   =|E  [g'(iLLl)] 

9, a  9,a 

=  ^E[g'(Y)] 


(1.12) 


=  -±-E[Yg(Y)] 


a  a  a 


=  i_E      ?[(x  -  e)h(X)]  , 

a        9, a 


where  in  the  third  equality  in  the  rhs  of  (1.12)  we  use  Lemma  1.1. 
In   subsequent  chapters,   however,  we   shall   require  a  multivariate 
version  of  the  above   identity  which  may  be   stated  in   the   following 
1 emma . 
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3h(X) 
Lemma   1.3     Let  X  ~  N(0,I    ).     Then   if  EQ[|  -%£-  |]  <  -,   for 

1  <  i   <  p,  one  has 


ah(X) 

Ee[-5xT-]  =  EeC(xi  "9i)h(*)]  -  (1'13; 


for  1  <  i   <  p. 

Proof:     Note  that  for  each  1  <  i   <  p, 

3h(X)  3h(X) 

hl~^~] "  ¥9iC"^r '  xi xi-i-xi+i v         (1-14) 

=EQEQ_[(X.-9.)h(X)    |   X1,...,X..1,X.+1,...,Xp] 


e  r(x,-9.)h(x)]  , 


where  for  the  second  equality  we  use  Lemma  1.2  and  the  independence 
of  the  X/s. 

An   interesting  consequence  of  Lemma   1.3  is  the  following:      If 
S(X)   =   (Xx  +  *1(X),...,   X     +  ♦   (X))   is  a  rival   estimator  of 
X  ~  N(0,I    )   for  estimating  9,   then  assuming   the   loss   (1.5)  with  Q  =  £ 

3*. (X) 

and  assuming  that  EQ[ |    ^~  | ]  <  »  for  1  <  i   <  p,   the  risk 

i 

difference  is  given  by 
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R(e,5)  -  R(e,x)  (1.15) 


V*I?.i -nr— tf.i»i<£»  ■ 


where   the  last  equality  is  obtained  from  Lemma    (1.3).     Hence   to 
obtain  estimators  5  which  dominate  the  usual   estimator  X  one  needs 
to  find  solutions  to  the  differential   inequality 


A(i}  =-  2^?=i  — hr~ +  ^i*i(i»  *  °  •  (1-16) 


2 
Put  <)).(x)   =  -a(x.   -   u . ) /  II x  -   u ii    ,    the   left  hand   side  of   (1.16) 

reduces  to  -  a(2(p"2)"a)   <  0  if  0  <  a  <  2(p-2),  p  >  3.     Hence,  for 
II  x  -  yll 

a(X  -  u) 
an  estimator  of  the  form  X ^  dominates  X  under  the   loss 

~     llX  "  u1 

(1.5)  with  Q  =  I    .     This  is   the  classical   result  of  James  and  Stein 
(1961). 
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1.4     The  Subject  of  This  Research 

In  Chapter  2,  we  consider   the  case  of  arbitrary  Q  and  V   (and  the 
loss  (1.5))   under  a   full   sequential    sampling  scheme.     We  show  the 
exact  dominance  of  a  class  of  estimators  over  the  usual  mean  vector 
in  estimating  q.     Further,  we  obtain  asymptotic  risk  expansions  for 
such  estimators  up  to  the  second  order  term.     A  Monte  Carlo  study  is 
also  performed  to  compare  the  risks  of  the  sample  mean  and  two  rival 
classes  of  estimators. 

In  Chapter  3  we  consider  a  multiple  regression  set  up.     Here  we 
wish  to  estimate  the  vector  of  regression  parameters  3  under  the 
loss   (1.5)  with  Q  =  Z'   Z  ,  Z     being  the  usual   design  matrix  of  order 

nxp,  and  V  =  (Z*   Z  )     .     Again,   the  sampling  is  done  sequentially 
with  each  additional   point  consisting  of  p  non-stochiastic  regressor 
variables  and  observing  the  response   (a  scalar).     Here  we  show  the 
exact  dominance  of  a  class  of  estimators  over  the  usual   least  squares 
estimator.     Further,  we  obtain  asymptotic  risk  expansions   for   such 
estimators  up   to  the   second  order  term. 

In  Chapter  4  we  consider  a  multivariate  version  of  the  Behrens- 
Fisher  problem  of  estimating  the  difference  of  two  normal  means  when 
the  variances  are  assumed  to  be  unequal.  The  sequential  analogue  in 
a  univariate  setting  was  first  introduced  by  Ghosh  and  Mukhopadhyay 
(1980),  and  was  later  generalized  to  a  multivariate  setting  by  Chou 
and  Hwang  (1984).  We  have  proposed  in  Chapter  4  an  extension  of  the 
stopping  rule   (1.7)    to   handle   the   two  sample   situation.     Also,    the 
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loss  (1.5)  is  used  in  the  estimation  of  v,    the  difference  of  two 
normal  mean  vectors.  We  show  the  dominance  of  a  class  of  estimators 
over  the  difference  of  the  sample  mean  vectors.  Once  more  we  produce 
an  asymptotic  risk  expansion  for  such  estimators  up  to  the  second 
order  term. 

In  Chapter  5  we  return  to  the  one  sample  problem  where  Q  is 

2 

diagonal  and  the  entire  covariance  matrix,  formerly  o   v,  is  assumed 

to  be  diagonal  and  unknown.  With  the  sequential  sampling  rule  (1.7) 
generalized  to  this  situation  and  the  loss  (1.5),  we  have  produced  an 
asymptotic  risk  expansion  (up  to  the  second  order  term)  for  a  class 
of  estimators  dominating  asymptotically  the  usual  mean  vector. 


CHAPTER  TWO 

SEQUENTIAL  SHRINKAGE  ESTIMATION  OF  THE  MEAN 

OF  A  MULTIVARIATE   NORMAL  DISTRIBUTION 


2.1     Introduction 


Let  X. ,  X?,    .    .    .  be  a  sequence  of  independent  and  identically 

distributed  N(0,a2V)   random  vectors,  where  ©eRP  is  unknown,  and  V, 

positive  definite,   is  known.     The  problem  is  estimation  of  0.     Based 

on  X1t    .    .    .    ,   X   ,    if  we  estimate  e  with  6     =  MX,,    .    .    .    ,   X   ),   let 

the  loss  incurred  be   (1.5).     We  have  seen  from  Section   (1.2)   that  for 

this  situation  the  sequential   sampling  rule   (1.7)  and  the  estimator 

X.,  can  be  considered  as  an  empirical  minimax  rule.     Consequently, 
~N 

producing  a  class  of  estimators  which  dominate  X^  takes  on  a  greater 
meaning  than  a  mere  pathology. 

In  Section  2.2  we  exhibit  a  class  of  James-Stein  estimators 
which  dominate  L,   the  usual  mean  vector,  and  simultaneously,  address 
the  two  objections  raised  in  Section  1.3.     The  first  objection  was  in 

reference   to  Ghosh  and  Sen   (1983-84)   and  their   two  stage   sampling 

2 
scheme.     Unlike  a   two  stage  procedure  which  uses  only  sm  in  defining 

the  stopping  rule,   the   stopping  rule   (1.7)   uses  an   updated  estimator 

of  a    at  each  stage  of  the  experiment,  and  thereby  demands  fewer 

observations  on  an  average.     The   second  objection  was  in  reference   to 

Takada   (1984)   and  his  version  of  the  James-Stein  estimator.     We  shall 
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see  later  in   this  section  that  Takada's  class  of  estimators  use  only 
p  of  the   (n-l)p  degrees  of  freedom  available  at  stage  n  to  estimate 
a2.     The  class  of  James-Stein  estimators  we  produce   use  all    the 
available  degrees  of  freedom  for  estimating  a2,  and  as  a  consequence 
allows  the  estimator  to  stabilize  for  large  n.     In  Section  2.3  we 
produce  an  asymptotic  risk  expansion   (as  c*0)   for  the  proposed  class 
of  estimators.     In  Section  2.4  we  give  the  results  of  Monte  Carlo 
simulations  comparing  the  risks  of  the  sample  mean,   the  proposed 
James-Stein  estimators  and  Takada's  estimators.     As  expected,  our 
estimators  achieve  greater  risk  reduction  than  Takada's. 

2.2     A  Class  of  James-Stein  Estimators  Dominating  X^ 

In  this  section,  we  consider  the  class  of  James-Stein  estimators 
^(X^...,^),  where 


6b(x1,...,xn)  =  sl b*n ,   , . —  a'YMn-  £>•    {2A) 

~n  ~1'       *~n         ~n         (^  -  X)'V  XQ  *V  1(%n -  M 


for  every  n  >  2,  where  s^  =  -^  s^(n  >  2),  b  is  a  constant  and  XeRp 

is  the  known  point  towards  which  we  want  to  shrink.     Very  often  X  is 

taken  as  the  prior  mean.     The  main  result  of  this  section   is  as 

follows: 

Theorem  2.1.     Under  the   stopping  rule   (1.7),  and  the   loss   (1.5), 
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r(£'{t2»£n)  <  R(~,a2'^N) 


for  every  be(0,2(p-2}) 
Proof:  First  write 


snLV^'~y~1(~V9) 


R(9,a2,6S)  -  R(9,a2,XN)  =  -2bE   ,[— ,  =r   q  . 3   (2.2) 

~    ~N     ~    ~N       0,a2  N(XM-  X)  V  XQ   *V  ^X.-  X) 

+  b2E   2[sJ/(N2{(XN-  M'V'W1^-  X)})]. 
9.0 


Since  Q  and  V  are  both  positive  definite,  using  the   simultaneous 
diagonal ization  theorem,   there  exists  a  nonsingular  D  such  that 

D0-1D'   =  I     and  DVD'   =  A,  a  diagonal  matrix  with  all   positive 

—    ~        ~P  — 

diagonal  elements.  We  write  ai  as  the  i  diagonal  element  of  A. 
Use  the  transformation  Z.  -  D(Xi  -M,  i=l,2,...  so  that  the  Zi '  s 
are  i id  N(s,a2A),  with  c  ■  D(9  -  X).  Writing  |n  =  n  Z.=1  Z.  for 
every  n  >  1,  one  can  rewrite 


s2  -  {(*-l)[p+2)rhnulU.-  Zn)'A_1(Zr  |n).  n  >  2   ;  (2.3) 


(X  -  XJ'V'^X  -  9)    =  Z    'A_1(Z  -  C),   n  >   1    ;  (2.4) 

~n    ~    ~      ~n    ~        ~n  ~      ~n    ~ 


(|n-  M'V"VV1(V  ~}   =  In  ^  L'  n  -l   • 


(2.5) 


21 


Then   from   (2.2),   one  gets 


R(e,a2.fi!j)    -  R(9,a2,XN)    =  -2bE  [(sJ/N)  (£„'  A"1^-  £)  )/(!,/  A"     |n)] 

~  ~  ~  ^  ,a 


(2.6) 


+  b2E       2[(sJ/N2)(ZN'A"2  Zn)_1] 


Next  we  use  the  Helmert  orthogonal    transformation  Y2  =   (Zj-  Z2)//T  , 

Y3  =   (Zx+  Z2-  2Z3)//6   ,...,   Yn   =  (Z1+  ...   +  Zn_r   (n-l)Zn)//n(n-l), 
...   to  write 

2         f/_    ,v/_.o\l"l    r-n         w    'a"1v         „    X    i  (2.7) 


s'  =  {(n-l)(p+2)}       E^g  Y.   A  AY.,   n  >  2 


where  the  Y.'s  are  iid  N(0,a2A).     Denote  by  Bn   the  a-algebra 


generated  by  Y2»    .    .    .    ,   Yp.     Now, 


E       2C(Sn/N)(Zn,A-1(Zn-  £))/(|N V  ZN)]  (2.8) 


CmE       2E       2[Hsn/n){In'.A     (In-S))/(Iny    Ir^WV 
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C»o 


■  ^E  2i»"l^I[^]*?.i»i  (A"»*iE.  2if-<r^h-)|Bn] 


A^E       2[n-2s^  (p-2)/(Z„'A-2  Zn,]    . 


For  the  second  equality  in   (2.8),  one  uses  the  independence  of  Zn  and 

(Y?,...,Y   ),  while  for  the  third  equality,  one  uses  Stein's  identity 
(of  Stein,   1981). 

Next,   note   that  for  every  n,   nZ"n'A~2  Zn/a     ~  ^=1Wni/a.  where 

2 

2  n^i 
the  W  -'s  are  independent  and  W  .  ~  X, ( — * — ),  i  =  l,....p.  Hence, 

co  a. 
using  the  result  (see,  e.g.,  Cressie  et  al.,  1981)  that  if 

P(U  >  0)  =  1,  then  E(U_1)  =  /J  E(exp(-tU) )dt,  one  gets 


E      oCnZV2  Z  /a2]"1 
5, a 


(2.9) 


=  /J    n  E[exp(-t  Wni/a.)]dt 
i=l 

=   /J    n  {(1  +  2ta~1)      2exp[-ntc2/(a2a.(ai+  2t))]}dt 

=  9p(Ar))  (say)    , 


where    a     =  A_(?,a  ).      In   this  section  we  need  only  that  gp(An)   is 
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nonincreasing  in  n.     Now,   using  again   the   independence  of  Z     and 
(Y2,...,Yn),   one  gets  from   (2.6)   -   (2.9), 


R(0,a2,5jj)    -  R(e,a2,XN)  (2.10) 


"bWp(An)Ea2[(Sn/n)(2(p-2)-7)I[N=n]] 


2 

*  -2b<P-2)  WpUn)E  2tn"lsn(1-  T"dm.]]    ■ 

a  a 


where  in  the  last  step  one  uses  0  <  b  <  2  (p-2).     Also,   in  the 

2 
above,   E  .  denotes  expectation  when   the  Y.   s  are   iid  N(0,a  A). 

a 
Accordingly,   for  proving  the  theorem,   it  suffices  to  show  that 


CmVAn)E  2[n_lsn(1-sn/a2)I[N=n]]   >-  °  for  a11    °*  >  °  •  (2"U) 


To  prove   (2.11),   first  observe   that  tr(QV)   =  trCQD^AD1-1)   = 

-tr- ( D ' ~ X QD — X A )   =  tr(A),   since  DQ~V    =  I   .     Let  nQ  denote   the   smallest 

-12  2 

integer  >  m  such  that  p(p  +2)     c  n/(tr  A)   >  a  .     Then,  we  write 
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lhs  of   (2.11)  (2-12) 

i  <2 

V1   m   ,.    sc      r„-l,2M    _   V 


2 

+  9p(An   )E  2[nolsn   (1  •T)I[N>nl] 
K       O     a  0  a  -  o 

sn+l 

+  Cn    {9p(An+i)E   2C(n+1)"lsn+l(1  '  JT1)I[N>n+l]: 
O     r  a  or 


Sn 
a  cr 


where  the  first  term  in  the  rhs  of  (2.11)  should  be  interpreted  as 

zero  if  nQ  =  m.     Note  that  for  n  >  nQ,  on  the  set  [N  >  n+1], 

s2  >  p(p+2)_1cn2/(tr  A)   >  p(p+2)_1cn2/(tr  A)   >  a2.     Accordingly, 
since  g   (a  )   is  nonincreasing  in  n, 

third  term  in  the  rhs  of   (2.12)  (2.13) 

2 

12     M  Sn+1, 


>CnVvi)E2[H'  wi-ir) 


■o  P         -      a 


s2 

n"lsn(1--7HI[M>n+l]3   ' 


Note   that  Ir»       n   is  a   B     measurable  function.     Then  using  the 
|_N>n+lJ  n 

representation  s2+1  =   ((n-l)s2  +  Un+1)/n,  where 
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IL,,   ■  (p+2)     Yl^A     L^  ~  <*  xn/(p+2)  independently  of 


,        ,     -     u;    ,.,   'H'      1A"1Yn.1     ~    crV, 

n+1         K  ~n+l~    ~n+l  p 


Y„,   ....  Y  ,  one  gets  with  probability  1, 
~2  '  ~n  3 


2 
•1.2      n    -J2±lu JB1  (2.14) 


E2[(n+l)-Xs^+1(l-^)I[N>n+1]|Bn] 


=  ^N^n+l]1         TnTTTTi  (ml)nV 


2 

=  I[.*i]''1,S"-7l 


*  I  rr.Wu1       ln-1)2.l    i    -2<.n'*  2(n-l)p         _  1) 

2 

pa     /        1  1        ,-■ 

2 


4  2  o 

+  !  rsn         3n-l  o     Sn       .      .    (n-l)p^  pa     .       n-1     i 


Mote  that  the  multiple  of  IrN>n+1]   in   the  extreme  rhs  of   (2.14)   is 

2 
a  convex  function  of  s  ,  where  the  minimum  occurs  at 
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n  ♦  (n'1)p 
2  ,  g2  Pjg       </,  (2.15! 

n  3n-l 


2         2 
Hence,  recalling  that  on  the  set  [N>n+1],   sn  >  a  ,   it  follows  that 


2nd  term  in  the  extreme  right  of  (2.14)  (2.16) 

rOn-Do2  .  2J  (n  +-^f]   +       o2(n-l)P     j 
-     CN>n+l]  nz(n+l)  n2(n+l)  (p+2)n2(n+l) 

Hence,  from  (2.13),  (2.14)  and  (2.16), 

third  term  in  the  rhs  of  (2.12)   >  0   .  (2.17) 

Next,  note  that  if  nQ  =  m,   since  Iru>n1]  =  l  with  Probability  1 

2  "  2 

ac     r  2n      V-,         2r  p         (m-l)p(  (m-l)p+2)  i    _       (m-l)po         2(     2) 
and  E  Jsjl-  -*■)]  ■  or  (--£_. ■ -*- — — ?— -}   -  -     — ■ 5— ~T  <^m  *' 


2^^  =  °  W"  (m-l)2(P+2)'     '   ~   (m-l)2(P+2)2 

>  0,   rhs  of   (2.12)   >  0.     For  nQ  >  m   (>  2),   first  note   that  for 

n  <  n     -  1,  on  the  set  [N  =  n],  s2  <  p(p+2)_1cn2/(tr  A)   < 

p(p+2)-1c(n  -l)2/(tr  A)   <  a2  so  that  using  9   (\)  nonincreasing  in  n, 


first  term  in   the  rhs  of   (2.12)  (2.18) 

>  g   (A     )"°       E  9[n_1s2(l  --r)IrN-nl]    • 

-  3p     n       n=m         2  n  02     LN-nj 
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In  view  of  (2.11),  (2.12),  (2.17)  and  (2.18),  for  proving  the  theorem 
it  suffices  to  show  that 

(2.19) 


To  prove  (2.19),  first  writing  s2  =  ((n  -2)s2  ,   +  U  )/(n  -1), 

no         o      o 
one  gets 


E2Kls*0(1-Tr>W]] 

a  o  a  ~  o 


(2.20) 


E2E2|:"o5n<1--r)ICN>n0]IVl] 

o     a  o  a  -  o  o 


2       _  4 


<».-»•!! -i+p$r*2    V2)2<-i+2(v2)<-i^  +  ^ 


E  2<C n   (n  -1) 


o_ 
o'no 


/       7722 

nQ(no-l)   a 


rCN>n0]^ 


"  E  2[K   Sn  -I"  bnn(sn  -1^]   +  "%   J    W]]   ' 
aOO  00  0-0 


where 
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tyg><yi-jk>    b    _  (v2)2     c     .     (v2)p 
"%      vv1)2  n°   no(v1)2 '  n°    wi)2(p+2) 

(2.21) 

(p-2)(n  -2) 
Let9n     =bn     "an     =n   (n-l)(p+2)    (>0)»   S0   that 

0  0  0  0       0 

c       -  g       =  2(no-2)/{no(no-l)(p+2)}  >  0.     Also,   let 
o  o 

2(n  -2)p 

d       =  2b       -  a       = — i —  ,  \ln.0\    so  that  dn     e   (0,1).     Now  rewrite 
\  no         no       VrV1,(p+2)  °o 


extreme  right  of   (2.20)  (2.22) 


4  4 

•    s       .  s„     . 

ihT  E  2^V(Snl-r  "H   +  gnJ-V  -  2<-l  +  *2> 


+  cn     "  V3lCN>njJ 


4 
s. 


0   a       0  0  cr  -   o 


■J.-. 


If  E  ^Tpr^s2,  _!  -  — V)T[N>n  ]]  -  °'  noting  again  that  Sn  <  ° 
a       o  o  o  -  o 

on   the   set  [N  =  n]   for  all   n   <  n  -1,   one  proves   (2.19)   from   (2.20) 

and   (2.22).     Otherwise  noting   that  dn     <  1,   one  gets  from   (2.19), 

o 
(2.20)  and   (2.22)   that 


29 


lhs  of   (2.19)  (2.23) 


>z"°_;2E2Cn-V(l-^)I[N=n] 


n=m 


2 


■12       ,,  o 


n  -1 


2UV     bn  -l^1  2~|X[N>n  -1]- 

a  o  a-o 


Proceed  inductively  to  get  either  lhs  of  (2.19)   >  0,  or  finally  end 
with 


2 

lhs  of  (2.19)  >  E  2[m"1s^(l  -  -y)I[N>m]]  >  0  .  (2.24) 

a  o 


as  calculated  earlier. 


2.3     Asymptotic  Risk  Expansion  for  SL  and  s^ 


First  we  obtain  an  asymptotic  (as  c  •*•  0)  expansion  for  the  risk 
of  JL.     Observe 


RfG.a2,^)   =  2cn*  +  cE  2[(N-n*)2/N]  (2.25) 

a 

V2  V2  *  2 

=  2c       a(tr  QV)       +  cE  2[(N-n    r/N]   . 
a 


a.s.  ^2  a.s.      2 

From  definition  N       +     ■»  as  c  +  0  so   that  s^       +     a    as  c   >  0. 
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2         2  4 

Also,   using  Anscombe's  Theorem,   /Np(?N  -  <j  )  +  N(0,2a   )   so  that 

—  L  2 

using   the  delta  method,   /Np(sN  -  <j)  •*>  N(0,  (1/2)CT   )• 

Next  use  the  inequality 


v2  v2„ 

(tr  QV/c)       sN  <  N  <  m  +  (tr  QV/c)       sN_1   .  (2.26) 


Dividing  all  sides  of  (2.26)  by  n  ,  letting  c  +  0,  and  using 

a.s.  *  a-s. 

s„   +  a  as  c  +  0,  it  follows  that  N/n    +  1  as  c  *   0.  Thus, 

N 

_  L         9  L         2 

/n*p(sN  -  a)  *  N(0,(l/2)a ),  and  also  /n*p(sN_1  -a)   *  N(0, (l/2)a  ). 

Again  from  (2.26),  one  gets  the  inequality 

(n*)  V2  (\-a)/a   <  (N-n*)/(n*)  V2  <  w/(n*)  1/2+(n*)  V2  (sM_ra)/a. 

(2.27) 

Since  m/(n*)   /2  >  0  as  c  +  0,  it  follows  from  (2.27)   that 
(N-n*)/(n*)   lz    *  N(0,  -L-)  as  c  ♦  0  which  implies  that 

1/  L  ,  a-s. 

(N-n*)/N  '2->-  N(0,  i-)  as  c  +  0  since  N/n*       ♦    1  as  c  +  0.     Also, 

the  uniform  integrability  of  (N-n*)2/N  for  all   c  <  cQ   (specified) 

can  be  proved  by  repeating  essentially  the  arguments  of  Ghosh  and 

Mukhopadhyay   (1980).     Thus  E  2[(N-n*)2/N]   >j-as  c   +  0,  and  it 

a 
follows  now  from   (2.25)    that 
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R(9,a2,XM)   =  2cV2a(tr  QV)V2+|-  +  o(c).  as  c  *  0   .  (2.28) 


Next,  we  find  the  asymptotic  risk  expansion  for  £N.     It  follows 


from  (2.6)  and   (2.8)   -  (2.10)   that 


R(e.o2.«J)  (2.29) 

■  R(9,a2,L)-2b(p-2)E  J(s2/H)j      n  (l^taT1)         expC^-fo^  a,  (a!+2t) )dt] 
~N  ac  0  i=l  a  1     n 

1/  S2 

+  bV*E     [(sJ/N)/"    S  (l^taT1)"    2exp(-J^i:P,la-(a^t))dt]  . 
ac       n         0   i=l  a  >      i 


,  a.s.     .         2  l/2a. s.  V2  V2 

As  c  -  0,  s£      -    a4  -P        and  Nc     "      ♦     (tr  QV)       *  =  (tr  A)       * 
(p+2)^ 


L/2 
(SP    a.)       °.     Hence,  for  £  t  0,  as  c  *  0, 


1/  c 

c-^sJ/Njf    n  (UZtaT1)"    2   exp(-^    ?    a   (a.l^t))dt  (2'30) 

N         o  i=l  ff    1=1     l     i 


p  V2    ,  -  V2 

n  (*■"■-      ■     x 

o  1-1 


1/  »        P  It  1      ' 

(sJ/(Nc  /2))/     ,n  (l+2tc      a.1) 


.2 


,        MtC      *      yP         1 )rit 

eXp(-  —j—  %x — )dt 


V2 
a.(a.+2tc       ) 
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2  V?  V? 

>  {— B-y  a4/(a(tr  A)        )}/"  exp(-  |  ( tr  A)       £?_,    C.2/a2)dt 
(p+2T  '"  0 

2       „  -V2  -V2    _       o         i 

=s-X_Ta4(tr  A)         (tr  A)         (if.    tf/a.)   i 
(p+2)2 

=       P         a4(tr  QV)_1{(9  "   *)'    V'W^e  -   X)}"1   , 
(p+2)2 


since  tr  A  =  tr  QV,  and 


-P     r2/,2  -»  r'a~2r  =  ^ fi-xrnWnvnT1fnvn' r1 


z?-iVai  =  £'&  5  =  (H),S'(23tB')    (2^  ]    S1-"^         (2,31) 


=  (0-x),v"1d"1d'"1v"1(o-x) 


=  (e-x),v"1Q"1v"1(0-x) 


Next  we  prove  the  uniform  integrability   (in  c  <  c  )   of  the  left  hand 
side  of   (2.30)  when   Z  t  0.     First  note   that 

lhs  of  (2.30)  U.Z2) 

2 


<  (sJ/(NO)[/Jexp(-^zP=lir7TiITrr)dt 

o  11 

2  _  £ 


I1    1  l 
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<    (sJ/(Nc))(K/N)    =  KsJj/(Nc        )2    , 


where  K  is  a  constant  not  depending  on  c.     In  deriving  the  first 
inequality  of   (2.32),  we   have  also  used   that  t/U^  2t)    +  in   t  for 

all   i   since  a.   >  0  (1=1,. ...p).     Thus,   it  suffices  to  prove  the 

1  4         \   2 

uniform  integrability  of  sj]/(Nc       )     in  c  <  cfl.     First  use  the 

inequality 

E,[(sJ/(N2c))I     ,     ,  ]   <dVE     [{sJ/(N2c)}1+5  ]  (2.33) 

a2       N  [s*/N2C  >  d]  a2       N 

for  some  s'efO,1/^).     Taking  5  in   ( 5 ' ,  V2  ) »  and  using  Holder's 
inequality,  one  gets 


E  ,r{s!/(N2c)}1+6   ]  (2.34) 

a2       N 

U+6)(l+fi') 


<[Eo(N2c)-(1+6)](1+6    )/(1+6)CE  ,(,*)         6"6         ](5"6    )/(1+6)    • 


2in    v-l  J  lu    2^H, 

a  o 


Next  observe  that  for  any  arbitrary  e  in  (0,1), 

I   2[(N2c)-'W»]  <  E  2[(N2C-(1+6,I[N<En,]+(,2n*2c)-(1+{):    (2.35) 
a  a 

<  (m2c)-(1+5)P  2(N<sn*)  +  (E2tr(QV)a2)-{1+5)  . 

a 
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To  proceed  further  we  shall  require  the  following  lemma, 


Lemma 


2.1  For  any  0  <  e  <  1  and  p  >  1,  P  9(N<en*)  =  0(c 


(V2)(m-l)p 


Proof:  Note  that 


). 


P  2(N<en*)    =  ^!|J*]   P  2(N=n)    , 


(2.36) 


where  [u]  denotes  the  largest  integer  <  u.     Now, 


P  2(N=m)  =  P  2Cs2m<  c-^j) 
a  o  — 


(2.37) 


di  2  ,  (tn-l)p  r      m       , 

=  P(x(m-l)p  -<  — 2—  C  *WT] 


Using   the  fact  that 


P(xk  <  d) -k- 

k  r(|+  1) 


(l/2)k/2dk/2e-d/2 

+P(x^+2<d) 


(2.38; 


k+2 

0   (dk/2)  +  0Q(d~2_) 
e  e 


0  (dk/2)   , 
e 


we  get 


P  2(N=m)   =  0g(c 


(V2)(m-l)P 


) 


(2.39: 
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Therefore  all  we  need  show  is  that 


[en*]  (V2)(m-Dp 

Z         ,    P  „(N=n)   =  0(c  )    •  (2.40) 

n=m+l     a2 


Now,   for  n  >  m+1, 


72  <  c       nL_x  (2.41) 


P^N-n)   <P^(sn  <  c^^) 

-  P(Y2  <   (n-l)P  c       "2    .) 

"  P(X(n-l)p  -         „2     C  tr(QV)J 


<     inf  exp(h  IZjU*.  c  TJ^)E[exp(-hx2n_1)p)] 


(n-l)p 

.  r         ,.    (n-l)p  „      n       w   1 x       2 

inf  exp(h 2—  c  ~nWy)(T^) 


h>0 


o  (n-Dp 

n  1    )        2 


r(n-l)pM  n         1    n,         n         1 

Wl— 2        (1   "  C   trTWT7)3(C   tFTQVT  a2 

(n-Dp 


=  Cexpd  -  c  — ^--j-Jtc  TFTQyT^3 


Lw.       n         l_n       2 


where   for   the   third  inequality  in   (2.41)  we   use   the   fact  that 

P(U<d)   =  inf  ehdE[e"hU].     Next  observe   that  for  n   <   [en*], 
h>0 

2       1  9 

c  — JLrrr^  <  e     <  1.     Since  x  exp(l-x)    is  increasing  in  x  for 
tr(QV)    g2  - 

0  <  x  <  1,   one  gets 
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(n-l)p 

a  —     o  — '     ° 

(2.42) 

(m-l)p  (m-l)p  (n-m)p 

(m-l)p  (m-l)p     _  (n-m)p 

<«     2       "iWV     2  £    n^'^xpU-V]     2 

Triyvj   a^  n=m+1 


The  series  in  the  extreme  rhs  of  (2.42)   is  convergent.     Therefore 


r     #1  (V2)(m-l)p 

E[en     P  ,(N«n)  =  0(c  )    .  (2.43) 

n=m+l     c 

o 


Hence,   for  m  >  2,   c  <  c     <  1, 


V     -  6 
rhs  of   (2.35)   <  K0(m2)-(1+6)c02  ♦  (e2tr(QV)a2f(1+5)    ,      (2.44) 


where  K  (>  0)   is  a  constant  not  depending  on  c.     Further,  using  the 
o 

2 
backward  martingale  property  of  Tn  =  {(p+2)/p}sn  and  Doob's  maximal 

inequality  for   submartingales,   it  follows   that  for  every  r  >   1, 

(2.45) 
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where  K,   is  a  constant  which  only  depends  on  r.     Combining 

(2.33)   -   (2.35)  and  (2.44)   -  (2.45),   the  uniform 

4       l/2     2 
integrability  (in  c  <  cQ)  of  sJ/(N       c)     follows.     Thus,  for  m  >  2, 

9  *  x  (i.e.,   z  t  0),  as  c  +  0, 


2 


n  -  1/p  £ 

c_1E    [(sJ/N) /"    n  (l^taT1)        exp(-  4  xP=1     ,(a;+2t) 

a  0   1=1  a  11 


^  -p      J )dt]        (2.46) 


2 

_P-^a4(tr  QV)_1{(0-X)'   V'VVVx)}"1 
(p+2)^  ~*  


Similarly,   it  can  be  shown  that 


2 


1  9.  ™       P  -1     "    '2  Nt       D  ^1 

c_1E  2[(sJ/N)/      n  (1+2*/)        exp(-  -J  x?=1  a.(a.+2t 
a  *  0   1=1  o  11 


N-*  rP       1 Mt]        (2.47) 


+  _P2(tr  QV)_1{(0-X)'   V'VV^G-X)}"1   , 


as  c  ♦  0.     Hence,  from  (2.28)   -  (2.29)  and  (2.46)   -   (2.47)   it  follows 
that  for  m  >  2,  and  0  t  x,  c  ■>  0, 

R(0,a2,6j)  <2-48> 

=  2c    2a(trQV)     2  +  i-  c  -  cb  — P— 7(2(p2-4)p"1-b)a2(tr  QV)"1 
~~  dp  (p+2T 

x  {(e-xjVVV^e-x)}"1  +  o(c)   . 
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For  9  =  A,   i.e.,   C  -  0,   it  follows  from  (2.29)   that 


R(X,a2,sJ) 


(2.49) 


2/M%r- 


=  R(X,aSXN)    -  2b(p-2)E  2(sN/N)/o    .n   <1+2tai    } 
a  i=l 


v2 


dt 


P  ,  -V2 

+  bV'E  2(sJ/M)/J  _n  (H^tap        dt 


2  -2r     ,„4, 

:  2 

a 


,-V2 


lv-1 


Note  that  Q  ,n  (^taT1)         dt  =  EU^W.aT1)     ,  where  the  W.'s  are 


i=l 


V?    a-s.     „       o  V2  n  -V2 

iid  x2.     Note  s2/(Nc       )     +    p^o  /(°(t<"  QV)       )  =  -fy  cr(tr  QV) 

-1/2 


1/2. a.s.         2 


V2 


/(Nc    2  )   V  _P         a4/(a(tr  QV)       )   =  -^—j  a3(tr  QV)  as 


(p+2)' 


(p+2)' 


c  ♦  0.  Hence,  using  the  uniform  integrability  (in  c  <  cQ)  of 

1/2  lll 

s2/(Nc       )  and  sJ/(Nc       )   for  m  >  2,  one  gets  from  (2.28)  and 
N  N 

(2.49)   that  for  m  >  2,  as  c  ♦  0, 


2     b  V2  > 

R(X,0\5°)  =  2c       cr(tr  QV) 


(2.50) 


V2  2  . 2 

c       ab 


£—'9  P-li-  b)(tr  QV) 


-V, 


(P+2! 


2(2       P 


x  E(EP=1W./ai)"1  +  o(c       )    . 
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Hence,   for  0  <  b  <  2(p-2),  m  >  2,  asymptotically   (as  c  *  0),   for 

k  _ 

9  t  X,    the  percent  risk  improvement  of  £^  over  X^  is 


100{l/2b— E— -{2(pZ-4)/p  -  b)o}(tr  QV)~J/'  (2.51) 

(p+2)2 


x  {(e-x)'V  XQ 


V2  V2 

,Y"1Q"1V"1(9-X)}"1c         +  o(c       )    . 


For  9  =  X,  0  <  b  <  2(p-2),  m  >  2,  asymptotically   (as  c  *  0)   the 
percent  risk  improvement  of  S„  over  X^  is 


100{V2(tr  QV)"1}  b— £-^(2(p2-4)/p  -  b)E(zJ _.W  /a   f1  +  o(l)   . 

-        (P+2r 

(2.52) 


Observe   that  the  dominant  term  in  both   (2.51)  and   (2.52)   is 
maximized  when  b  =  (p2-4)/p.     Unlike  the  fixed  sample  case,   the 
optimal    choice  of  b  in   the   sequential    case  depends  on   unknown 

parameters  9  and  a2.     From  an  asymptotic  point  of  view  as  evident  in 

2 

(2.51)  and  (2.52)   it  appears  that  for  small   c,  b  =  (p  -4)/p  is  the 

optimal   choice,  which  is  different  from  p-2.     We  may  also  note  that 

the  a.'s  are   the  eigen-values  of  A  =  DVD*,   i.e.,    the  eigen-values  of 

d'oV  =  QV.     For  Q  =  V  =  I    ,   the  expressions  given   in  (2.51)  and 

(2.52)  simplify  respectively  to 

2  V2  V2 

50b— 2_(2(p2-  4)/p  -  b)ap"3/2{(e-X)'(e-X)}'1c       +  o(c       )        (2.53! 
(P+2T 
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and 


50b— £— j(2(p-  4)/p  -  b)?'1^)'1   +  0(1)    .  (2.54) 

(P+2P 


Remark.     It  should  be  noted  that  in  defining  £N  in   (2.1),   if  one 

uses  s2  instead  of  s2,   then  carries  out  the  asymptotic  calcula- 
n  n 

tions  in  the  same  way  as  before,  one  will   find  that  the  optimal 
choice  of  b  =  p  -  2.     Then,   however,  our  method  of  proof  does  not 
give  an  exact  dominance  result  as  in  Theorem  2.1. 


2.4     A  Monte  Carlo  Study 


For  simplicity,   consider  in  this  section  the  case  when 

X  =  0.   Q  =  V  =  I    .      In   this   special    case,  Takada   (1984)   has   shown 
~      ~    ~      ~      ~P 
that  if  one  defines 


%  -  w\  !*•  »>- 2  •  (2-55 


then  the  estimator  sjj     =  5^  (X^,...,^),  where 


5b     =  x     -_^L_  X  (2.56) 

~**       *       nX'    I     ^ 
~n  ~n 


dominates  X..  for  all   0  <  b  <  2   (p-2).     The  difficulty  with  the 

~N 

estimator   n  is   that  even   for  large  n  it  does  not  stabilize   since  for 

every  n   >  2,  n     ~  (p+2)-1x2.      In   this  section,   our  objective   is   to 


41 


compare  the  risk  performance  of  X^,   5^  and  5^   ,  with  b  =  p-2  and 
b  =   (p2-4)/p. 

For  Monte  Carlo  simulation,  we  take  p  =  3.     Also,   in   this 
section,   for  actual   risk   simulation,  we  consider   6^       and   6.,       or 

.    2 

St/-*"*  and  y-4)/P,  where  s„b  -  5^,. .  .,*)   -   (1  -  -^-)+  X 

~N  ~N*  ~n       ~n  -1  ~n  nX     X        ~n 

~n  ~n 

n  >  2,   6bn     =  &bn    (X......X)   =   (1  -  -r^H  +  X       n   >  2. 

~n*      ~n*  -1  ~n  nX   'X        ~n 

~n  ~n 

In  the  above  a+  =  max(a.O).     Such  plus  rule  versions  of  James-Stein 

estimators  prevent  overshrinking,  and  in  fixed  sample  situations, 

perform  better  than  the  usual   James-Stein  estimators. 

To  simulate   the   sequential    sampling  procedure,  and  evaluate 

the  estimators  under  consideration,  a   large  pool   of  trivariate 

N(0,I3)  variables  was  generated  with  e'   =  (0,0,0),   (0,  \  ,  \  )» 

11 
(0V?»7P'    (0»1'1)»    U*1*1)*    (0./2./2),    (0,1,2).     Also,   c,   the  cost 

per  sample  unit,   is  taken  as  c  =  .01,   .05,   .1,    .25,   .5  and  1.     It 

should  be  noted  that  the  risk  of  the  estimators  5^  and  6^     depends  on 

9  only  through  lien,  while  X..  has  a  risk  which  does  not  depend  on 

neii.     For  each  fixed  9  and  c  under  consideration,  from  the  pool, 

samples  were   taken   sequentially  from  the   top  down   until    1000 

experiments  were  completed. 

A  single  experiment  would  be   taking  sequential    samples  from  the 

pool    until    the   stopping  criterion  was  met.     At  this  point,    sampling 

would  stop,    the  number  of  samples  would  be  recorded  and  the  estimators 
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v        .p-2      .p-2      Jp2-4)/p      Jp2-4)/p  .    ..    .  .    .     .  . 

~N'  S     '     N      *     N  '   £n  '  their  associated  losses 

are  computed. 

On   the  completion  of  1000  experiments,  we  compute  the  average 
losses  for     L,   6J"2,   «<P2-»)/P     6j>-2     &(P2-4>/Pf  and  these  are   the 
simulated  versions  of  the  corresponding  risks.     Also,  at  this  point, 
we  compute  the  percentage  loss  improvements 


100(L(e,L)   -  L(0,6P_2))/L(0,X..); 


2 
100(L(9,XN)    -  L(9,6^P  _4)/p))/L(0,XN) 


100(L(9,XN)    -  L(e,5^"2))/L(9,XN); 


2 

100(L(9,XN)   -  L(0,S,iP  "4)/p))/L(0,L)    . 

Our  simulation  findings  are  summarized  in  Table  2.1  and  Figures 
2.1  to  2.6.  It  is  clear  from  the  table  that  as  in  the  fixed  sample 
case,  when  X  =  0,  the  risk  improvement  of  all  the  estimators  is  most 
substantial  when  let  =  0,  and  the  improvement  keeps  diminishing  as  D9i 

moves  further  and  further  away  from  zero.  Also,  it  is  clearly  evident 

2 

that  with   the  proposed  stopping  rule  b  =   (p  -4)/p  does  better   than  b  = 

p-2.     This  is  also  clear  from  the  asymptotic  risk  expansion.     Also,   for 
a   fixed   11011/O  as  c  decreases,   i.e.,    the  average   sample   size  gets  larger, 
the  percentage  risk   improvement  decreases  as   in   the   fixed  sample  case. 
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The  opposite  is  the  case  when    neu  =  0.     The  main  reason  is  that 

-     2  "  -V2        2 

when   X  =  0,   NiiXj.il     behaves  as  a  multiple  of  c         H0ii     when 

9^0,  while  NiiX^n2  ♦  xp  when   9  =  0. 

Figure  2.1  plots  the  risk  of  the  sample  mean,  X^,   the 
James-Stein  estimator  at  b  =  p-2,   <5P~  ,  and  Takada's  estimator  at 

b  =  p-2,   Sp~2 ,  versus   H9ii,  for  six  levels  of  cost.     As  can  be  seen 
~"*  ~ 

from  the  graphs,   sP"2,  for  each  level   of  cost,   has  smaller  risk   than 

X„  while,  as  was  expected,   sj"     has  smaller  risk   than   Sp     .     Also,  we 
can  see  that  the  risk  difference  is  greatest  at  nen  =  0  and 
diminishes  as   H9ii  moves  away  from  zero.     Figure  2.2  plots  the  percent 
risk  improvement  of  6Jj"2  over  XN>  and  5Jj~     over  XN>   versus  cost,  for 
four  levels  of   191.     Here  we   have   the  percent  risk   improvement  of 
Sp,~2  being  better  than  that  of  s[!~2 ,  for  each  level   of   iei.     Also,  we 
see   that  the  percent  risk  improvement  is  greatest  at  iiqii  =  0  and  in- 
creases as  c  decreases,  while  for   nen  /  0  the  percent  risk  improve- 
ment decreases  as  c  decreases.     Further,  for  fixed  cost,  the  percent 
risk  improvement  decreases  as   11011  moves  away  from  zero.     Figure  2.3 
plots  the  risks  of  the   sample  mean,  jjL,    the  James-Stein  estimator  at 

b  =  (p2-4)/p,   sf,p  "4)/p,  and  Takada's  estimator  at  b  =  (p  -4)p, 

2 
5(p  -4)/p     versus    nen    t   for  tne   Six  previous  levels  of  cost.     The 

patterns  and  conclusions  for   these  plots  are   identical    to   those  of 


Figure  2.1.     Figure   2.4  plots  the  percent  risk  improvement  of 

2  2 

6(p  -4)/p  over  7       and   s(p  -4)/p  over  x       versus  cost,   for   the   four 
~N  ~N  ~N,  ~N 
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previous  levels  of  neu   .     The  patterns  and  conclusions  for  these 

plots  are  identical    to  those  of  Figure  2.2.     Figure  2.5  is  a  plot  of 

2 
the  risks  of  L,   6P~2  and  &lP  _4>/P     versus   neu,  for  the  six 

previous  levels  of  cost.     This  figure  gives  us  an  idea  of  how 

Figures  2.1  and  2.3  compare  as  far  as  the  James-Stein  estimators  are 

concerned.     Here,   for  the  first  time,  we  can  graphically  see  that 
o 
(p  -4)/p  has  smal-|er  risk  tnan  6^P~2)    (while,  as  before,   both 

2 
dominate  X\,).     Hence,  we  have  more  evidence  in  favor  of  (p  -4)/p  as 

the  optimal   choice  of  b.     Figure  2.6  plots  the  percent  risk 

2  2 

improvements  of  S^"2,   5P~2,   6^  "4)/p  and  6<p  "4)/p,  over  XN,   versus 

cost,   for  two  levels  of   neu.     Here,   for  the  first  time,  we  can 

2 
graphically  see  that  5^  P  has  the  greatest  improvement  while 

2 
(p  -4)/p   .     second     6P-2  is  third  and  6p"2  has  the  least 

improvement  over  XM  giving  once  again  more  evidence  in  the  favor  of 

2 

(p  -4)/p  as  the  optimal   choice  of  b. 
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Figure  2.2  Percent  Risk  Improvement  at  b=p-2. 
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Figure  2.3  Risk  Analysis  at  b=(p  -4)/p. 
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Figure  2.4  Percent  Risk  Improvement  at  b=(p  -4)/p. 
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Figure  2.6  Percent  Risk  Improvement  for  All  Estimators. 


CHAPTER  THREE 

SEQUENTIAL  SHRINKAGE  ESTIMATION  OF 

LINEAR  REGRESSION   PARAMETERS 


3.1     Introduction 


Consider  the  linear  regression  model  Y.   =  z^.3  +  e^   i=l,2,..., 

where  the   e.'s  are  independent  and  identically  distributed 

N(0,ff2),    B(pxl)   is  unknown,  and  z,,   z^,    .    .    .  are  known  pxl 

vectors.     Let  Z^  =  (z^...  ,zr|)  and  Y^  =  (Y^. .  ..YJ  ' .     Assume 

that  the  rank  of  Z'    Z     is  p.     The  problem  is  estimation  of   S.     Given 

Y   ,   let  the  loss  incurred  in  estimating   8  by   5  (Y   )  be 
~n  ~        ~n  ~n 


Ln(5n*B)  *h*n  '  s)'(z!   Z  )(5    -  S)  +  en   .  (3.1) 

n~n~        n  ~n       ~      ~n  ~n     ~n       ~ 


The  usual  estimate  of  8  is  the  least  squares  estimator 

6     =   (Z'    Z   )     Z'    Y„,  which  is  distributed  N(6,a  (z'    Z   )      ),  with 
an         ~n  ~n       ~n  ~n'  ~        ~n  ~n 

risk 


R(B,a2,B „)    =  ELn(L,S)  (3.2) 


=  |e[(6  -   B)'(Z*   Z    )(B-    6)]  +  en 
n         ~n     ~       ~n  ~n     ~n     ~ 
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2 

H-+  en   . 
n 


a2    1/2 
The  above  risk  is  minimized  at  n  =  n*  =  (-^-)       .     However,   if 

a2  is  unknown  there  does  not  exist  any  fixed  sample  size  which 

minimizes  (3.2)   simultaneously  for  all   a  .     Again,  motivated  from  the 

optimal   fixed  sample  size  n*,  we  propose  the  following  sequential 

procedure. 

P  MSE      1/2 
N  =  inf  {n  >  m  :   n  >  (— r-2-)       }  <3-3) 


where  m  >  p+1  is  the  initial   sample  size  and 


MSE     ■   I  Y     -  1     I     ™2/(n  -  p)  (3.4) 

n  ~n       ~n  ~n 


for  each  n  >  p+1.     The  above  stopping  rule  was  first  considered  by 
Mukhopadhyay  (1974)  who  proposed  the  estimator  6N  for  £  and  studied 
asymptotic  properties  of  N  and  8^. 

Our  next  step  is  to  show  that  the  above  procedure  can  be 
motivated  from  a  minimax  criterion.     With  this  end,  first  let  £  have 

a  N(u,t2I   )  prior.     Then  the  posterior  distribution  of  8  given 

~      ~P 

Y     =  y     is 

~n       in 

27'(t27  7'   +  aZl   )~l(v     -  Z  u).  (3.5) 


"(»  +  'In^lnln  +  al~n]     (^n  "  ^}* 
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^L-^2^  +  »Vln> 


(3.6) 


Next  we  prove  the  following  matrix  lemma, 
Lemma   3.1 


Z>2^  +  ^n'"1  '   (^n  *  "V"^ 


Proof:     Use  the  identity 

Using  Lemma  3.1  it  follows   that 


:2Z'(T2Z  Z'    +  a2!)"1!^   -  Z   u)  (3.7) 

^n       ~n~n  ~n         ~n       ~n~ 


<*  &„  +  °  y  z~;(in  -  Jn«» 


r  T 


-iiP+4(^'"1>"1(Jn-';' 


Also,   using  Lemma  3.1, 
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I  -  t2z'(t2z  Z'  +  o  In)  Zn 
ip    ~n   ~n~n    ~n   ~n 


(3.8! 


I  -  (t2z'Z  +  o2l    )_1(t2Z'Z  ) 
~p     ~n~n    ~p     ~n~n 


•  lp  -  (A&  *  "'ip)"1!^  *  «%  "  •  %' 


2,  27.7  t  2T  v-1 
=  *  <T  znzn  +  a  ip) 


2        2     . 

£_jZ'z  +£_  i  )_1 
^£n£n    2  V 


^Ip^^)"1)"1^)"1 

T     K     T 


Thus  from  (3.5)  and  (3.7)  -  (3.8)  it  follows  that  the  posterior 
distribution  of  3  given  Yn  =  y n  is 


r  T 


(3.9) 


^(ip^f^'"1'"1^'"1'- 


The   loss   (3.1)   being  quadratic,   the  Bayes  estimator  of  3  is 


sS-s^ip*-2?^)"1)"1^-^  . 
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with  risk  ^trCdp^lZ^r1)-1]*  en. 

Next,   consider  the  sequence  of  fN(v,mI   ),  m>l}  priors  on 
Then  the  Bayes  estimator  becomes 


2 

6B  =  V  +   (I      +^-(Z:Zn)"1)"1(£n   "  ^    ' 
~n       ~        ~p      m      ~n~n  ~n       ~ 


2  2 

with  risk  -2-  tr  [(I     +  ^-  (Z^f  V     +  en.     Here,    the  Bayes 

2  .2 

risk  converges  to  ^-  +  en  as  m  ♦  °°,  and  R(M  ,£n)  =  —  +  en 

for  all   3.     Hence  n*.   the  sample  size,  and  £n>   the  estimator,   is  a 
sequential   minimax  rule.     Consequently   (3.3)   can  be  viewed  as  an 
empirical  minimax  stopping  rule. 

In  Section  3.2  we   show  the  dominance  of  a  class  of  estimators 
over  Bu  for  the  loss  (3.1)  and  the  stopping  rule   (3.3).     In  Section 
3.3  we  provide  an  asymptotic  risk  expansion  for  £N,  and  an  asymptotic 
risk  expansion  for  the  shrinkage  estimators  dominating  £N  both  up  to 
the  second  order  terms. 

3.2     A  Class  of  James-Stein  Estimators  Dominating  £N 

In   this   section  we  consider   the  class  of  James-Stein  estimators 
5]J(YN)  where 
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bsn 
6b(Y    )    =   6     --s = (!„    "   X)  (3.10) 

^^        ^      (s-  x)'(z;z  )(8  -  X)  ^      w 


for  every  n  >  p+1,  where  s2  =-£^MSEn,  b  is  a  constant  and  XeR     is 
the  known  point  towards  which  we  shrink.     The  main  result  of  this 
section  is  as  follows: 

Theorem  3.1     Under  the  stopping  rule   (3.3)  with  m  >  2p,  and  under 
the  loss   (3.1), 


R(6,a2,6^)    <  R(£,a2,£N) 


for  every  be(0,2(p-2)), 
Proof:  Note  that 


R(8,a2,6j)  -  RtB.a2,^)  (3.11) 


b2E   ci 2j ] 

£.-2¥(£N-^'^N)(£N-x) 


4 

2bE    [i  — r-T*—- —  (£n-  y,(&,(ii-  *)] 
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h2c4 

1  b  s„ 

E       E  Til  " 


£.*'    "  %'l^^l^in^ 


bs2 


'U-*wv3<v*'WVp}w 


To  proceed  further  we  shall   require  the  following  lemma  concerning 
the  orthogonal   decomposition  of  SSEn(=(n-p)MSEn)   in  regression. 

Lemma  3.2     Let  Yp  =  Zn£  +  en,  where  en  ~  N(0,^In)   for  every  n   >  p+1. 

Assume  that  (Z'Z   )   to  be  invertable  for  every  n  >  p+1  and  define 
~n~n 

SSE     =   ii Y     -  Z  5  n2.     Then  the  following  results  hold. 
°°  n         *t\       ^n~n 

(i)         ln+l^n+1  =  ^i~n  +  -n+l^n+l   ; 

ijji        (71      7       \"1  =   (7'z   )_1(I     -  — z     ,z'    ,(z'Z   )"   ),  where 
(n)       (4n+14n+1)       "  lfcn^n'     ^p       "    ~n+l£n+lVfen~n; 


nn  =  1  +  ^+i(^Zn)"ISn+i   ! 


<"*>     Sn+1  =  i  +  tf'XVVl'  where  en  =  Vl  "  Vl  5n  and 


B     =   (Z'Z   )_1Z'Y      ; 
~n         ~n~n       ~n~n 


<iv>       SSEn+l  -  SSZn+J^T 

n 
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(v)    Y  -  Z  8  is  distributed  independently  of  e  . 
K    '         Ln       ~n~n  n 


Proof  of  (i):  Note  that  Zp+1  =  (Zn£n+1)'-  Then 


In 

Z'      Z         =  (Z'z     ,)[        ■      ]  (3-12) 

£n+l~n+l       v~n~n+rL     - 

£n+l 


■  luh  +  £n+lVl 


Proof  of  (ii):     Using  the  matrix  identity  from  Rao   (1965)   that  for 
any  matrix  A  (pxp)  and  (pxl)   vectors  u  and  v, 


(A  +  uv'f1   =  A"1 1  (A^Uv'A-1),  (3.13) 

~      ~~  ~  1+v'A     u     ~ 


then 

i^iW1  -  <&.  *  ^i&i1"1  (3a4) 

1+£n+ll£n£nj     Sn+1 


^^'^k-^^l^l11^'^ 
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Proof  of  (iii):  Note  that 


£n+l  =  (Li+l£n+l}  £n+lln+l 


(3.15) 


=  l&n^Ip  "f  2n+l2i+l(&.rl,(5i!n  +  Vl^l' 


"  (Ip-^^n)"1^!^!^ 


+  Vl(2^n>"1£n+l(1-^5i+l(2^rl5n*l) 
"  fin  vf^^W^lin1 


+  7Vl(arlJn+l 


£n  +  ^1«n(^,"15n+l  ' 


where  in  the  fourth  equality  in  the  rhs  of  (3.15)  we  use  the 


definition  of  n  . 


Proof  of  (iv):     Note  that 


SSEn+l  =  '!n+l  "  in+lSn+11' 


(3.16! 


In  "  £n£n+l 


Yn+1  "  ~n+l~n+l 
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!|In  "£rin  "   Chl^lnl^l^/ 


+  (Y„.,-  znXlR zB,.   Z  Z         z     -) 

n+l     ~n+l~n       n    ~n+l  ~n~n       ~n+i 


2 

»Yn    "    *„£,/    +  "7  "2   "^Cl  < ZnZn  )  "^n  ( VZn£n  ^ 

~n   ~n~n     c         n  ~n+i  ~n~n   ~n  ~n  ~n~n 


e_ 


n 


SSEn+4(vl)  +  (£n-^vl))2 

•Vi  n 

2 
e 

SSE     +  —  , 

n        \ 


where  in  the  third  equality  in  the  rhs  of  (3.16)  we  use  part  (iii), 
while  in  the  fifth  equality  we  use   the  definitions  of  n    and  e   . 


Proof  of   (v):     Since  Y     -  Z   g     and   e     are  both  normally  distributed 
~n       ~n~n  n 


then  to  prove  their  independence  we  need  only  show  that  their 
covariance  is  zero.     With  this  end  note  that 


Cov(Yn  -  Z   a   ,en)  (3.17) 

~n       ~n~n  ~n 


Cov(Yn  -  Zri5n,Yn.1-Z*    ,Bj 

~n       ~n~n  ~n+l  ~n+i~n 

-  Cov(Y   -z'   A)  +  Cov(Zns„,z'    .0) 
~n  ~n+l~n  -ji~n  ~n+l~n 
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Variyyz^z^  +  Zn  Var(Sn)zn+1 


2 
Note  that  if  we  define  Xj_  =   lY     :  -  Zp+1£p+1'     and 

2 
e  •        i 
x.   =     1+p-1   ,  1=2,...,n-p,   then  from  Lemma  3.2  the  X.'s  are 

1       ni+P_1 
independent,  and  X.  ~  o2Xy  1=1, ...,p.     Also,  from  Lemma  3.2,   for 

each  n  >  p+1, 


SSEP+i  =  "Vi  "  ^iP+i"2  =  xi  ' 


2 


2 
SSEn-l  =  'In-2  "  In-lln-/  +  ^T  =  Xl+X2+-"+Xn-P-l   ' 


and 


'n-2 


2 
s*n  =  'In-1  "  Zn-lln-l"2  +  ^  "  W"-+Xn-P-l+Xn-p 


Consequently,  MSE     =   (n-p)~1SSEn  can  be  considered  as   the  mean  of 

2 
(n-p)   independent  and  identically  distributed  x:  random  variables. 

Denote  by  B     the  a-algebra   generated  by  X^,   .    .    .    ,   xn-p*     Tnen 

using  Lemma   3.2, 
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c         r 0 I    1  (3.18) 

E„  _2C7T— ^T^TTTTT  CN=n]J 


^  ^n'V   ^nZn)(£n-M 


s* 
E   ?E   2^ .  .    » ICH-n3|Bn:i 


E   ?[sJlr,_n1E   J-x ;   /   , |B  }] 


■V^^Vvy'^^-y1" 


where  for  the  second  equality  in   (3.18)  we  use  the  fact  that  I[N=n] 
is  a  function  of  X^   .   .   .   ,  Xn_p  only,  and  in   the  third  equality 

we  use  the  independence  of  X^ Xn_p  and  £n. 

Next,   note  that  for  every  n  >  p+1,   since  Z^  is  positive 
definite,   using  the  diagonalization  theorem,   there  exists  a 
nonsingular  Dn  such  that  ^(Z^)"1^  =  A"1,  a  diagonal   matrix  with 
all   positive  diagonal   elements.     We  write  ani  as  the  1-th  diagonal 
element  of  A  .     Use  the  transformation  L     =  D'(S  -  X),   so  that 

L     ~  N(£  .^A"1),  with  i     =  D"(B_-  x).     Then 
^n  ~n       ~n  ~n       ~n  ~n     ~ 


E       ,[-, £—„ 1  -  E         2C-t^-]    •  (3.19) 
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U^n  „ 

Here,  for  every  n,  r~    ~  ^-lWn1 ,  where  the  W^'s  are  independent 


e 


and  w   „  x2(^^ ),  1-1 p.  Hence,  using  again  the  result 

ni         l   2a2/ani 

that  if  P(U>0)   =  1  then  E(U_1)  =  /"  E[exp(-tU)]dt,  one  gets 


kn£n~n 

Ihs  of   (3.19)   =  E         J p— ]_1  (3-2°) 

~n' 


rQ  E[exp(-tzfalWni)]dt 

L  ii  E[exp(-tW   .)]dt 
ui=l  ni 

f«  P_  r  ^    i     .  ^2  gni  2t  UHt 

'o^W     exp(-I7^-^)}dt 


/0{TT2t)P/2exP(-lfetT2-Z?=lani?ni^ 


/J^/^e^^l     (£-x)'(Z;Zn)(8-X))dt 


VV    lsay)  ■ 


We  shall  next  show  that  g  (Afl)  is  nonincreasing  in  n.  Note  that 
for  any  nxp  design  matrix  Z  , 
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11 
~n~n 


E"         7 

i'=lzli 


En     z 
L i-1  P-1.1 


En    lZl, 
1=1    ll 


En      z2 

i=lzli 


Ei=lzp-l,i 


E"        7        7 

i-=lzlizp-l,i 


En       72 

'i=lzp-l,i 


(3.21) 


Then  for  any  a  e  R   , 


2'<^n'2'£?.l£j=l«i(^n>ua; 


(3.22) 


1<J 


E"=itai  +  Vik  +  •••  'Vp-u1' 


which  is  a  nondecreasing  function  of  n.     Hence,  using   (3.21)  and 

(3.22),  q  (A  )   is  a  nonincreasing  function  of  n. 
3p     n 

Next,  observe  that 


V2  (v^'(z;zn)(Bn-x) 


^n-i)'(Z^n)^n-£)ICN=n]] 


(3.23) 
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2 


'£„!»' (^n)(V£» 
E  ,ts'I,„  ,]E   ,[- 


,2  "c«  B.«2(i;-i)'(z;?n><!n-i> 


by  the  independence  of  X^  .  .  .  ,  Xn_p  and  £n»  and  that  I[-N=n]  is 
a  function  only  of  X.,  ...  ,  X    . 

Using  the  diagonal ization  as  before, 


L^V^ 


(3.24) 


E    J 

En'*   Lnirin 


i=lE    2[7TT— Lni(Lni  ^ni)ani] 
5„>*  LALn 


i  =i 

in ' "  ~n~n~n 


P  r    r  2  3    ni  -! 

Ei-iE      o[ct  "an — ; ] 

E  ,a     9  ni  LA  L 

Zn  ~n~n~n 


Sn'°  ininin   'tnintn' 
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2 
(p-2)E         ?[— 2 3 


(p-2)gp(An) 


For  the   third  equality  in   (3.24),  one  uses  Stein's  identity,  while 
for  the  last  equality,   one  uses   (3.19)  and  (3.20). 
Combining   (3.18)   -   (3.20)  and   (3.23)   -   (3.24), 

R(S,a2,6^)   -  R(£,a2,£N)  (3-25) 


2         s2 

=  bCmVAn)E2^FL{b4-2(P-2)}I[N=n]^ 
a  a 


2     s2 

<2(p-2)b  z;=mgp(An)  e2£M-£-i}  i[N3n]]  . 

K  a  a 


where  in   the  last  step  one  uses  0  <  b  <  2(p-2).     Accordingly,   for 
proving  the  theorem,   it  suffices  to  show  that 


2     s2 

Wp(ln»E  2CTT  ("I  "  ^[N-n]1  S  0  **  all   o2  >  0  .  (3.26) 

r  a  a 


With  this  end, -let  n     be   the  smallest  integer  >  m   (>   2p)   such 

-12  2 

that  p(p+2)       c  n^/p  >  a   .     Then,  write 
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s2     s2 

ihs„f  (3.26)  -  i"i  jpiv^r  «7  -  ^w-i1  (3-27) 

s2       s2 

'V-'yv1-?"111^",)' 

s2        s2 


0  "  ° 


2       2 
sn   rSn 


-9p(An)E2[TT4-l}l[N>n+l]]    • 


a  a 


where  the  first  term  in  the  rhs  of  (3.27)   should  be  interpreted  as 

zero  if  n     =  m.     Note  that  for  n  >  nQ,  on  the  set  [N  >  n+1], 

s2  >  p(p+2)_1cn2/p  >  p(p+2)_1cn2/p  >  a2.     Also,  by   (3.21)  and  (3.22), 
n  -  ~  ° 

q   (A   )   is  nonincreasing  in  n.     Hence, 
Jp     n 

third  term  in  the  rhs  of  (3.27)  (3-28> 

Note  that  IrN>n+1]  is  a  Bn  measurable  function.     Then  using  the 
representation  s2+1  -  ^JLj-  {(n-p)s2  ♦  ^  Xn_p+1},   one  gets  with 
probability   1, 

2         s2 
rsn+l   f  n+1  _  .i.  .B   i  (3.29) 

E  2[7T7T~  [~T      ltICN>n+l]|UnJ 
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(n-P)2s:+2(n-p)s^02+3-^^ 


=  Wl]C  (n+l)(n-p+l)V 


(n+l)(n-p+l)     J 


s       s 


2-2  2s4 

r,    1       (n-p)  '       - 

iW^rr  (n+1_p) 

p 


n  I  n  -  il  +  T  r(   1       (n-p)       .  1)     n 

=  I[N>n+l]  —  ^      l>       ^Wl]11^  (n+1.p,Z       n'7 


,      (n-p)((n-p+l)-2-*2-)       1     2 
—   (— — —  -  — )  s 

n+1  (n-P+l)2  n     n 

((n+l-p)-3^-) 

"^      c„*i-pi5     *    3' 


Note  that  the  multiple  of  IrN>n+1]  in  the  extreme  rhs  of   (3.29)   is 
a  concave  function  of  sn>  where  the  maximum  occurs  at 


(n+l)(n+l-p)2-n(n-p)((n+l-p)-2-£2-)     2         2 
s'   =  , s — t a      <   a      .  (3.30) 

n  2((n+l)(n+l-p)  -n(n-pp) 


2         2 
Here,  recalling  that  on  the  set  [N>n+1],  sn  >  a  ,   it  follows  that 


2nd  term  of  the  extreme  rhs  of   (3.29)  (3.31) 

,       ,2         ,      0  ,      (n-p)((n+l-p)-2 -£*■)       ,     2 

.  T  r,   1       (n-p)       _  Ua2  _   ,   1 __£!£_-  i)ff« 

S  ^Mjn+n^nTTT^T      n}«         (1*T  (n+1.p)2 
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1     (("+H0-3pk>     p    g2] 

^    („+l-p)z    p+2 


-  ^^h^t^i^3  pfe  -  f^"13^  *2 


<  o  , 


since  n   >  2p.     Hence,   from   (3.28),    (3.29)  and   (3.31) 


third  term  in  the  rhs  of  (3.27)   <  0  . 


(3.32) 


Next,  note   that  I[N>m]  ■  1  with  probability  1  and 


2       2  2 

E     [J!  (^  -  1)]  .  ^[(^)2  ^(m-P+2)   -  -By] 
a  a 


np-vp+2      m-p 


p+2- 


(3.33! 


=  £_{   P   )2_L-[2  -|(m-p)] 
rn^p+2'     m-p  p 


<  0   , 


since  m  >  2p.     Thus,   if  nQ  =  m  then  rhs  of  (3.27)   <  0.     For  nQ  >  m, 

2_12 
first  note  that  for  n  <  nQ  -  1,  on  the  set  [N=n],   sp  <  p(p+2)     en  /p 

<  p(p+2)_1c(n  -l)2/p  <  a2  so  that  using  the  fact  that  9p(\)   is 

nonincreasing  in  n, 


the  first  term  in   the  rhs  of   (3.27) 


(3.34) 


2       2 

n  -1  sn  s 

<  g   (A   )   z  °       E  ,[-2.  (-4-  UIrN_nl] 
-  yp     n       n=m       g2  n         2  LN-nj 
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In  view  of  (3.26),   (3.27),    (3.32)  and  (3.34),   for  proving  the  theorem 
it  suffices  to  show  that 


2         2 
2       2  s„       sr 


^^"'W  +  ^'X^W-0-     (3.35) 


2     1  2 

To  prove  (3.35),  first  writing  sn  =  7pr^{n0-1"P)sn  -1 

0    0  o 

+  ^2Xn0-pL  one  getS 


s2   s2 

Eg2[nQ  (a2    1}I[N>n0]] 


2         2 

sn       Sn 

o   ,     o 


■^7-"^V 


ivi-p'^-i^v^i-iP^2*3^2'4 


t2a - ; — ±r 


o     o 


«v1-p,s5  -i+^2a2 


rTTn-rpT  }l[N>nQ]] 


>n  -1 


E2CK    Hr--CnnV1_  V}I[N%]1    ' 
a  o     o  oo  o  o 
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1     (V1"P)2  1     (V1-P)(no-P-2pS2)     „n, 

where  bn     = -J ?-,  c       = 2 - .and 

m  L  (%-P-3  pfc     p     a  NQte   that 
no       no         (no-p)2      ^ 


S     "bn     =^^7[l-2^]<0  (3.37) 


o  o       n0ln0-p, 


and  that 


n     +(cn     -b„   )-— i-"ZC2(p-l)(no-p)-3-PT-(p-2)]>0. 


'o  "o        "o        n0(nQ-p) 


since  n     >  2p.     Also,   let 
o  - 


(3.38) 


nQ-l   (nQ-l-p)    (nQ-p-  -) 


(3.39) 


So  that  f       e(Otl).     Now,  rewrite 
o 


extreme  right  of  (3.36)  (3-40) 


2  2 

sn   -1  Sn   -1 
sE2[{(n0-l)(2bn-cn)7r4I   (     y     -1) 

a  ooo  a 
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n  -1 
o       o        a  o  ooo  o 


s2        s2 
sn  -1  sn  -1 


^fn£2[n-l(     2       '  1}I[N>n   1]    ■ 


o  a       o 


s2     i    s2     , 
n  -1     n  -1  2         2 

If  E  2[n  -1   (     2 1H[N>n   ]]   -  °'   n0ting  again   that  Sn  <  °     °P 

tj       0  a  -  0 

the  set  [N=n]  for  all   n  <  n  -1,  one  proves   (3.35)  from  (3.36)  and 

(3.40).     Otherwise  using  f       <  1,  one  gets  from  (3.35),    (3.36)  and 

no 


(3.40)   that 


lhs  of  (3.35)  <3-41) 

2  2 

n  "2  s2  s2  Sn  -1  sn  -1 

<  C     E  2[JU4  -  1)  W  ♦  ^2^-7"  "  l»W>n0-l]3   ' 


Proceed  inductively  to  get  either  lhs  of   (3.36)   <  0,   or   finally  end 
with 


2     $2 

lhs  of   (3.35)   <E2[^(-J-l)I[N>m]-]   <0  (3.42! 

a  a 


as  shown  earlier. 
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2  1 

Remark.      Initially  we  defined  S     =   (n-D)+a     MSEn'   reall'zl'n9   tne  need 

for  a  biased  estimator  of  a     in    5     from  Chapter  2.     The  exact 

~n 

dominance  was  then  proven  with  conditions  on  a     established  along 

2 
the  way.     The  particular  choice  of  a     =  -r<n-p)  was  most  appealing 

since   then  S2  = \ MSEn  =7£r-A-MSEn  and  hence 

n       ,       , .   i ,       >         n       p+d  n-p         n 
(n-p)+-<n-p) 

E  «[S  ]  =  -Ej  a  .     Therefore,   the  bias  would  be  negligible  for  large 

a 
p  (as  in  Chapter  2).     As  a  consequence  of  our  choice  we  needed 

m  >  2p,  and  thus  we  require  such  a  large  initial   sample  size. 


3.3     Asymptotic  Risk  Expansion   for  B.,  and  6., 


In  this  section  we  first  obtain  an  asymptotic  risk  expansion  for 
SM  up  to  the  second  order  term.     Subsequently,  we  find  an  asymptotic 
risk  expansion  up  to  the  second  order  term  for  5... 

Observe   that 


R(£,a2,BN)    =  pa2E[N-1]  +  cE[M]  (3.43! 


2cn*  +  cE[itpii] 


=  2c1/^pa2)1/^cE[l^W] 


a.s.  a.s.     „ 

From  definition  N       +     °°  as  c  ♦  0  so   that  MSEM       +     a     as  c  +  0, 

since  MSE     is   the  mean  of  n-p  independent  and  identically  distributed 
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, 2  L 

random  variables.  Also,  using  Amscombe  s  Theorem,  •'N-p  (MSEu  -  cr  )  * 

4  , l/9  L 

N(0,2o   )   so  that  using  the  delta  method,  /N-p  (MSE^-  a)  ■* 

N(0,  V2°2). 

Next  use  the  inequality 

(p/c)  1/2MSEN1/2<  N   <  m  +   (p/c)   /2MSEN^    .  (3.44) 


Dividing  both  sides  of   (3.44)   by  n*,  making  c  ♦  0,  and  using 

^2   a.s.  a.s. 

MSEN         *     0     as  c  *  0,   it  follows  that  N/n*       *     1  as  c  ♦  0.     Thus, 

, V2  L  V2  L 

/n*^  (MSEN     -  or)  +  N(0,  V2  CT   ) ,  and  also  /n*^|T  (MSEN_1   -  0)   * 

N(0,  V2ct2). 

Again  from  (3.44),  one  gets  the  inequality 


v2  .       v2 

lk    MSEM  -  s   M  n*    m       V2  MSEW  .  -  a 

— ^ <  ^  <  -=-  +  n*  — ^ ,        (3.45) 

V2     V2 
n*    n* 


V2  L  V2  L 

implying  that  (N-n*)/n*    *  N(0,  V2  )  as  c  +  0.  Hence  (N-n*)/N    * 

a.s. 
N(0,  V2  )  since  (N/n*)   +  1  as  c  *  0.  Also,  the  uniform 

(N_n*)2 
integrability  of q for  c  <  c  (specified)  can  be  proved  by 

repeating  essentially  the  arguments  of  Ghosh  and  Mukhopadhyay  (1980). 

2 
Thus,  E  9[   »   ]  *  V2  as  c  +  0,  and  it  follows  that  from  (3.44) 


a2L   N 


as  c  *  0, 
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V?  */2 

R(e,a2,£N)   =  2c        (pa2)        +  V2  C  +  o(c)    .  (3.46) 


Next,  we  find  the  asymptotic  risk  expansion  for  £N.     It  follows 
from  (3.11),    (3.18)   -   (3.20)   and   (3.23)   -   (3.25)    that 


R(B,a2,5JJ)    =  R(8,a2,B    ) 


(3.47) 


2b(p-2)E  2[^r   (^^Pi-^T^^'^^N^HB-Mldt] 


+  4E  2^T  (T^^xpt-^T^fi-x)'^^)}^)}^  . 

a       a  0  a 


Now,   for  technical   reasons,  we  shall   handle  the  risk  expansions 
separately  for  the  cases  8  j4  X  and  B  =  X.     However,  both  cases  will 


a.s 


a.s. 


require   the  facts   tha 


4  *'*'     4,   p   ,2       2  a*°-     2,   p   . 
t  as  c  *  0,   st       ♦     a  (rfc)   .   S,       >     a  {--) 


p+2'    '      N 


V2   a.s.         ?    V2 
and  Nc  *     (p<?  )       .     Also,  at  this  time,  we  shall  make  the 

following  assumption   that-^Z^)  ♦  K  as  n  +  -,  where  K  is  a 

1  a,s' 

positive  definite  matrix.     Hence  -f^Z^)       +    K  as  c  +  0. 

Consider  first  the  case  B  +  x.     Here,  as  c  +  0, 


=''?r  ^n*M-h^i-irKVi^n-iw      »-48» 
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n— v> 


V2    o  V2 

Nc  l+2tc 


a . s .       „4 


72       f     Nc 

exp{- 


V2 


a"  1/2 

1+2  tc 


L^£-x)'{^(Z^N)}(s-x)}dt 


';*  -J. (   P   )2  fexpl-t  (P%)        (B-X)'K(8-X)}dt 

l/2    P+^  0  or 


\ 


(pa2) 


T4_P 


(p+2T    (B-X)'K(B-X) 


Next  we  prove  the  uniform  integrability   (in  c  <  cQ)  of  the  left  hand 
side  of  (3.43)  when  B  t  x.     First  note   that 


lhs  of   (3.48) 


(3.49) 


Xrl  ..._r_  Nt  lo  ,virl/7' 


i^expt-^B-Xj'^Z^JKB-XjJdt 


0  3a 


+  exP{--^(B-X)'{^Z;ZN)}(B-X)}   /"   (^    /2  dt] 
3a 


<_n £ , 

'  n2c  (£-^'{|(z;zn)}(b-x) 


where  K?  is  a  constant  not  depending  on  c.     Thus  it  suffices   to  prove 


the  uniform  integrability  of -^-C(B-X) '  {-^(Z^)}  (S-X)]"     in  c  <  cQ. 

N  c 

First  use   the   inequality 
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s4  (3.50) 

E^l-^'fez^lB-A)}-1!     4 

N  c 
4 

i'  -.SM        r..    ...  rl  ......     ,!,„    ,,ria+« 


<d-5E,c(^-{(3-x)'^z;zN)}(B.x)r1)1+0] 


for  some  $'    e  (O,1^).     Taking   6  e  («' ,  V2  )  and  using  Holder's 
inequality,  one  gets 


E  ,[&-  K'-y^^ins-iir1)1*'']  "-si) 


<   [E   JN2c)-(1+5)] 
a 


(1+5) (1+6    ) 
.1  5-5'  (5-6*  )/(!+■«) 


x  CE  2(sJi(£-M'{|(z;zN)}(£-x)}-1)      °-°         ] 

CT 


Taking  6°  e   (6,  V2  )  and  using  Holder's  inequality  once  more  one  gets 
rhs  of  (3.51)  (3-52) 

<CE?(N2c)-(1+6)](1+6')/(1+5) 


(1+6) (1+6')  1+6°      6°-6 

,  4  5-6'  5°-6    nl+6° 

x   {[E  ?(sj)  ] 

en 


81 


(1+0(1+6°)     i+L     5-6' 

xCE2{(£-X)'{|(Z;ZN)}(£-X)}  6-6*  ]^°}^- 

a 


Next  observe  that  for  any  arbitrary  e  in   (0,1), 


E  2[(N2c)-(1+6)]   <  E  2[(N2cr(1+6Vn*]  +  (^cf  ^+»]     (3.53) 
a  a  - 

<   (n,2c)-(1+5)   P(N<en*)   +  (eW(1+6)    . 

To  proceed  further  we  need  the  following  lemma. 

Lemma  3.3  For  any  0  <  e  <  1,  p  >  1  and  m  >  p+1,  P(N<en*)  = 

(  V2  )(m-p) 
0(c        ). 


Proof:     Note   that  P  9(N<en*)   =  ^!"*]  P  ~(N=n)  where   [u]   is   the 
largest  integer  <  u.     Now 


P  „(N=m)   =  P  2(MSE     <  ^m2)  (3.54) 


nl   2       ,  m-p  c       2, 


(  1/2  ) (m-p) 
-  0e(c  )    , 


since  P(xj~  <  d)   =  0   (dk/2).     Therefore,   all   we  need  show  is   that 
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r     *i  (V2)(ni-P) 

E[en*     P  9(M-n)  ■  0(c  )    .  (3.55) 

n=m+l       2 


Now,  for  n  >  m+1, 


c     2, 


P     (N=n)   =  P  2(MSEn  <jt\c)  (3.56) 

o  o 

=  P(x2       <  J1ZP.C     n2 

vxn-p  -     p       2 

o 


<  inf  exp[h-^.^-n']E[exp(-h  x|-     )] 
h>0  p     o 


n-p 

•    r  ru    n"P    c      „2 -. , 1 »    2 


h>0 


2               2         ^£- 
rn-p,,       n     c   ,n,n     c   »  2 
cpC-^d 2-)Kp 2] 


2  2  ^- 

r        /i        n     c    wn     c    n   2 

=  [expd  -- 2)(^ 2)] 

K     a  a 


2  ? 

Note  that  for  n   <   [en*],  £-\<  e     <  1.     A1s°.   x  exp(l-x)   is 

o 
increasing  in  x  for  0  <  x  <  1.     Therefore 


2  2  ^=£- 

l[en*J  P  ,(M-n)   <   zJ^Cexpd  -  ^^)(^-^-)]   2  (3.57) 

n=m+l       2  -     n=m+l  p       ^     p       ^ 


m-p  m-p  I 
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m-p  m-p  n-m 

<c2     (eJL_)   2     ,,-  nm-P[exp(1  .  e2)£2]   2 

pa 


0(c(V2,Im-'")   , 


m-pr       , ,         2N  2- 
i     K[exp(l  -  e    )e    . 

using  Lemma  3.3  for  m  >  p+3,  and  c  <  cQ  <  1, 


since  E°°       .   n     p[exp(l  -  e   )e   ]         is  a  convergent  series.     Hence, 
n=m+l 


lhs  of  (3.53)  (3.58) 

<«,Ar'"'t'%H,'PlMSVrlwl 

-      O  0  0 

<K(m2)-<1+6'c0V2"6     *IeVl',WI   , 


where  K  (>  0)   is  a  constant  not  depending  on  c.     Further,   since  MSE 
o  " 

can  be  expressed  as  a  mean  of  independent  and  identically  distributed 
x?  random  variables  then  MSE     is  a  one  sample  U-statistic  with  kernel 
of  degree  one  and  hence  is  a  backward  martingale.     Consequently, 
using  Doob's  maximal   inequality  for  submartingales,   it  follows  that 
for  every  r  >  1, 


E2^-E2^MSVr  (3-59' 


<  E  ,[MSEJr 


34 


<  E  2[sup  MSEn]r 
o     n>m 


<  ^E  [MSE/  . 


where  K.   is  a  constant  which  only  depends  on  r.     Next 

observe  that  if  a     ♦  a  then  for  all   e  >  0  there  exists  an  mQ 

such  that  for  aJJ_  n  >  m  ,    Un~a|    <  e  implying  that  for  all   n  >  mQ, 

JL<     1        Let  U3(l^')(l+5°)      .     -  (£-M'{|(z;Zn)}(B-X), 
an       a-e  &_&>  n  n     n  n 

a  =   (B-X)'K(e-X)  and  e  >  0.     Then 


(l+fi')(l+fi°) 

E2Ci(£-i)'tJ<zfei)H£-i)}        °'6"  3  (3-60) 

a 


^•q  '[n^]  +  k  I[N>mo])U] 


s2uvtwu]+E/twu]1 


n<m       n 
o 


independent  of  c.     Combining   (3.50)   -   (3.60),   the  uniform 
integrability  in  c  <  c     of  4-{(£-X) '  \hl'l   )  }(£-£)  }_1  follows. 
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Thus,   for  m  >  p+3,  £  /  X,  as  c  ♦  0, 


''**  it/r  (Tin)  /2  exp(- ^-^S-M'te.lKs-Mldt]       (3.61) 


0 

4       p 
+  cr    — t 


(p+2p   (B-X)'K(8-X) 


Similarly,   it  can  be  shown  that,  as  c  +  0, 


Sk,     .»  i  P 


^  2^1"  <TOt>   n  »><-  \lWl-i»'li<^N»K»-i>l«]       (3.62) 


a"  0 

.2     1  1 


+  a 


p+2   (8-X)'K(6-X) 


Hence,   from  (3.46)   -   (3.47)  and   (3.61)   -   (3.62)   it  follows  that  for 
m  >  p+3,  3   f  X,  as  c  +  0, 


R(8,a2,^)  (3.63) 

=  2c1/2(pa2)1/2  +  V2c  +  cba2_P         L (b-2^-)   +  0(O    . 

(p+2P   (e-X)'K(B-X)  p 


Consider  now   the   case  6   =  X.      It  follows  from   (3.47)    that 
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2 

SM      .-         1  P, 


R(B,a2*b)   -  m,ad,y   ~  2b(p-2)E     [^  /     (^    "  dt]  (3.64) 


4 

+  4 E  «#  r  «rare>P/2  dt] 

o       o  U 


Note   that  /"  (TT2t}   ^  dt  =  p=y     HenCe'   Us1ng   thS   uniform 


2  A 

—  and  — 
He  *  NC/2 


integrability  of  — —  and  — j 

Nc  /2  Nc' 

from  (3.46)  and  (3.64)   that  as  c  ♦  0, 


in  c  <  c     for  m  >  p+1,  one  gets 


V?  V2 

R(B,a2,6b)    =  2c        (pa2)  (3.65) 

V2    n3/2  1  D2-4  ^ 

+  c      -E 7  a  -iy  b(b-2  V1)   +  0(c       }    ' 

(P+2)2       P"2  P 

From   (3.63)   it  can  be   seen   that,  asymptotically   (as  c  ♦  0)   the 
percent  risk  improvement  of  6..  over  B^  for  £  t  X  is 


/2  2  .        V2  V2 

10o{l/2b-P— y 2 {b-22-±)}c       +o(c       ),         (3.66) 

(p+2T    (P-X)   K(B-X)  P 


while  from  (3.65),  asymptotically  the  percent  risk  improvement  of 
6..  over  6M  f or  8  =  X  i  s 


87 


100  { 1/2  b 


(p+2) 


2  p-2 


(b 


l± 


)}  +  o(l) 


(3.57) 


Here  the  dominant  term  in  both  (3.66)  and   (3.67)   is  maximized  when 

b  =  (p2-4)/p.     Therefore,   from  an  asymptotic  point  of  view  as 

2 
evident  in   (3.66)  and  (3.67)   it  appears  that  for  small   c,    (p  -4)/p 

is  an  optimal   choice  for  b   (as  in  Chapter  2). 


CHAPTER  FOUR 

SEQUENTIAL  SHRINKAGE  ESTIMATION  OF  THE  DIFFERENCE 

OF  TWO     MULTIVARIATE  NORMAL  MEANS 


4.1  Introduction 


Let  X,,  X-,    .    .    .  and  Y  ,  Yg,   .   .    .  be  two  independent  sequences 
of  random  vectors  with  the  X. 's  independent  and  identically 
distributed  Nte.^fy.)  and  the  Y^s  independent  and  identically 
distributed  N{92>a|y2).     Here  the  e^s  e  RP  are  unknown  while  the 
V.'s  are  known  positive  definite  matrices,   1=1,2.     The  problem  is 
estimation  of  y  =  Q^  -  ©2.     Based  on  Xp   .   .   .   ,  Xn  and  Y^,   .    .    .    , 

Ym,   if  u   is  estimated  by  «n>||  «  ^(X^..  .,Xn;  ^ Y^)   let  the 

incurred  loss  be  given  by 


L(e,,e0,6      )  =  (6        -  u)'Q(8    m  -  v)  +  c(n+m),  (4.1) 

~r~2'~n,m  ~n,m      ~    ~  ~n,m 


where  Q  denotes  a  known  positive  definite  matrix  and  c(>0)  denotes 
the  known  cost  per  sampling  unit.     The  most  natural  estimator  of  v  is 

Vm^n   -!mwithrisk 

^I'^'VVW  (4-2) 


38 
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E       2  2[(^,m-ii),2LW;i)]  +  c(n+m) 

2l'S2'crl'or2 


=  E  2     2[^n-Si)'2^n-£l)   +   (!n1-22),S(!rn-S2) 

91»£2»CTl'a2 


+  2   (X     -  QJ'QlL  "  9o)]  +  c(n+m) 
~n      ~i     ~  ~m      ~t 

2 

=  _L  tr   (Q  V, )   +  —  tr(Q  V0)   +  c(n+m)    . 
n  ~  ~1        m  — £ 


2  2 

a2 


When  a2  and  a2  are  known,   the  above  risk  is  minimized  at  n  =  n*  - 

a?  V2  4  1/2 

(-i-tr(Q  V,))         and  m  =  m*  =   (-=- tr(Q  VJ)       .     Also,  for  this 

a^tr(Q  Vx)    V2 
pair   (n*,m*)  we  have  n*/m*  =  [-* ]       .     However,  when 

G2tr(2  ~2} 

a?  and  a?  are  unknown  no  fixed  sample  size  will   minimize   (4.2) 

2  2 

simultaneously  for  all   a,   and  o2.     In  this  case,  motivated  from  the 

optimal   fixed  sample  size  n*  and  m*,  and  additionally  by  n*/m*,   the 

following  sequential   procedure  is  proposed  to  determine  the  sample 

sizes. 


T  =  N  +  M,  where 


v2 


N  -  inffn   >  nQ:    (J3L  tr(Q  V^)       } 
and  <4-3> 
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M  =  inf{m  >  nQ:   (J2-  tr(Q  Y2)) 


y? 


n     >  2  and  observe  X     ,   or  Y     .if  neither  process  has  stopped  when 
o  -  ~n+l        ~m+i 


2  ,r~2 


v. 


n/ra  <  [sj  tr(fl  V^]       /[s;  tr(Q  Vg)] 


v2 

n/m  >  [s^   tr(Q  Vj)]       /[s*   tr(Q  V2)] 


V2 


(4.4) 


respectively.     Here, 


,-1  „n 


and 


s<-  [(n-l)pr1  ^=1(X.   -Xn)V%   "V 


-[(-"-DP^^Xj-y'v^Xj-  V   • 


(4.5) 


for  every  n,m  >  2.  The  above  stopping  rule  is  a  multivariate 
analogue  of  the  stopping  rule  proposed  in  Ghosh  and  Mukhopadhyay 
(1980).  Similar  stopping  rules  were  considered  by  Chou  and  Hwang 
(1984)  who  estimated  u  by  X^  -  YM- 

A  point  of  interest  concerning  the  above  stopping  rule  is  that 

*n         V2 
if  N=n  is  the  first  occurrence  of  n  >  (—  tr(Q  V^)    while 
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~2 

Sm 


v2 


m  <   (JH  tr(3  VJ)         then  necessarily  we  must  observe  Ym+1  since 

-P  V2     ~2  V2 

n/m  >  Cs;  tr(fl  V^]       /[s;  tr(fl  V2)]       . 

To  see  how  the  above  sequential   sampling  procedure  can  be 

2 
motivated  from  a  minimax  criterion  let  £.   have  a  Nf^.^.B.)  prior 

i=l,2.  Then  following  the  arguments  from  Section  1.1  the  joint 
posterior  distribution  of  C^.y '  given  ~i  =  ~i'  i=1>---'n  and 
V*J'   j=1 ml'S 


N( 


II  +  Up  +^T^i5l1)_1(5n  "  Il} 


m 


lz+  Up+-TW21,"1(im-221 

mT_ 


n  1 


A  A      -1       -1 

r      mx_ 


Therefore,   the  posterior  distribution  of  u   =  ^   -  £2  becomes 


1     u  o-lx-1 


1*-1, 


NUri2  +  Up  +-rWl l    (5n1Il,-lIp  +T7W2  }    (vi2 
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2  2  2  2 

<?1  ai         _i         _i         a9  °9        -1         -1 


Hence,   the  Bayes  estimator  of  v  becomes 


with  posterior  risk 


2  2  a2  a2 

fHtrVip  -rfiV1  +#y^  ♦AsV1" +  c(n+m) 


The  posterior  risk  being  independent  of  any  unknown  parameters 
implies  that  the   sequential   Bayes  rule   is  a   fixed  sample  rule  with 
sample  size  determined  by  minimizing 


2  2  2  2 

*toJrhS\*  +^t  BA)_1  +  #  V!*  +-tbA)_1"  +  c(n+m) 


with  respect  to  n  and  m. 

Next,   consider  the   sequence  of    {N(y.  .IB.. ) ,   1   >  1}  priors  on 
9.,   1*1,2.     Then  the  corresponding  sequence  of  Bayes  estimators  for 
u  is 
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2  2 

5B       =   y,-Y9  ML  +  —  V1B71)"1(Xri-Y1)-(I      +  —  hb1)'1^-^    • 
-ji.m       ~1   ~2         ~p       n-|    ~1~1  ~n   ~i       ~p       m-|    ~£~£  ~m  ~^ 

with  risk 

2  2  2  2 

*ukr Wk  +  J fi  V1  +  f- Wk  +  7 BA)_1  >]  +  c(n+m)  • 

The  latter  converges  to  K  =  j~-  trfQV^)  +  ^  trtQvy  +  c(n+m)  as  1  ■*», 

and  R(Q,,%, o?,o?,W       )   =  K  for  all    0       GL.     Hence,   under  the  loss 
~l~212~n,m  ~i      ~c 

(4.1),   the  fixed  sample  rule  with  sample  size  determined  by 

A         A 

minimizing  —  trtQV^   +  ^p  tr(QVg)  +  c(n+m)  with  respect  to  n  and  m, 

and  estimator  W       ,  is  a  minimax  rule.     Consequently,   the  stopping 
'-n  ,m 

rule   (4.3)  can  be  interpreted  as  an  empirical   minimax  stopping  rule. 

In  Section  4.2  we   show   the  dominance  of  a   class  of  estimators 
over  W.,  „  for  the  loss   (4.1)   under  the  stopping  rule   (4.3).     In 
Section  4.3  we  develop  an  asymptotic  risk  expansion   for  W^  „  and  for 
estimators  belonging  to  the  above  class  up  to  the  second  order  term. 


4.2     A  Class  of  James-Stein  Estimators  Dominating  W^  M 


In   this  section  we  consider  the  class  of  James-Stein  estimators 

^M>2(VVwhere 


brb 
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b  s2 
SV\l   ,Y    )   =    (X     -  Y"    )    -   [ *I"  Q-V^X     -    X  )  (4.6) 

~n       ~i  v, 
2Sm  (f^:1^  -   XJ] 


:  r"i    JL2  lim       ~2 

milLr,  "   *2'v2 


for  every  n,  m  >  nQ  (>  2),  where  s^  «^sn,   sm  -  -&%  s"m,  b., 

p  i   -1  -1  -1  - 

1-1,2  are  constants,    nX^  -  ^  «v     =  (^  -  ^)  ^  Q     Vj_   (X^  -   X^, 

"S>  -  v?2  =  <!■  -  ^'viYV^  -  &- and  v i=1'2  are  the 

known  points  towards  which  we  shrink  X^  and  Y^,  respectively.  The 
main  result  of  this  section  is  as  follows: 

Theorem  4.1  Under  the  stopping  rule  (4.3)  and  the  loss  (4.1), 


for  every  b^b^  e  (0,p-2). 


Proof:     Note  that 


Kh'&'i'i'Zu* 2)  ~  R{^^  v^Vm1  (4-7) 


•2 
s. 
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-2 


2   22' 4  M"lM  '   ^2"V2 


.4                                                  -4 
+  b*E         ,[-, J *-]  +  b^E         2C-2-^ T3 

Sl-4  n  »x^  '  h\  ^'a2M  'Im"  ^2"v2 


-  2b,b„E  0     -,[- 


"2  -2 

SN  SM 


)1D'3L  9       9L 7  -  2 


•<?N-   h^AV^"    ^2)]    ' 


Here,  by  using  Stein's  identity  as  in  the  proof  of  Theorem  2.1  we 
have 


"2 

E       2E         $N      2    ib'h^b-h"  (4'8) 

9^  NIX,,  -    X1nyi 

-2 

=   a?(p-2)E         gC-y-z " T~]   * 

2l'«L  N    l|X^  "   *l\ 


Similarly, 

-2 


Si^ —  fY..  -    JU'v"1^,  -    Go)]  (4.9) 


E         2[v-  2     <?H-^'Vi(!H-5z)] 

®2'CT2  M,Im  "    *2"V 


'2 
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?                                      SM 
=   <£(p-2)E         9\.-y-_ — 1 

%'°2M    %  ~   V'V2 


Therefore,   using   (4.8)  and   (4.9) 

rhs  of   (4.7)  (4-10> 

=  -2a?(p-2)b.E         ohr-^ H   "  2a2(p-2)b  E  gC-yr-* T] 

1       1^7%r^  S-2m2|1Wv2 

-4  -4 

o  SM  ?  SM 

+  b2E         [      _  N      2  ]  +  b|E       2C— r] 


Ivi  H  "VV^  %'a2  M  Wv 


"2  -2 

.  SN  M 

2b, DpE  p     ? 


^.^oJ.^MiX^  -    ^1}         MIY,,-    Xgly 


•«?M-    ^'V~lVYi1(?M-    ^)] 


From  the  Cauchy-Schwarz  inequality  we  have  for  any  two  vectors  a^ 
and  b,    |a'b|   <    nan    iibn,  where    1. 1  denotes  the  usual   Euclidean  norm. 
Then,  using  the  additional   fact  that  there  exists  a  nonsingular 
matrix  P  such  that  P  Q~V   =  I_,  we  have 

-2  -2 

_  $H . J *-  (L   -    JU'vTV^Yu  -    X0)         (4.11) 


2-  •  -■     "     2    ih  -  *i]  h  Q~  V<Lm  "  *2> 

""  M    II V  _        A        II 


NllX^   -    X^       MilY^  -    X^ 
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"2 


lw-1, 


t-    \  .1  £~%Lih-k»l 


-2 

-5l_.p-1v:1(?1i-  h,n 


"%-  Vv 


*%-  Vv, 


~2   KW     ~2' 


2 


<  c- 


-][- 


MV   ^"v       "V   Vv, 


Consequently,   using   (4.11) 
rhs  of   (4.10) 


(4.12) 


;-2  -sz 

<  -2d?(p-2)b.E         9C-5 H   "  2<^(p-2)b2E         .[-^ 2~] 

-     ilp     l^7ivv5        2       2^4MVVv? 


-4 

SN  n      .      u2r  r  SM 


"4 


+  b^E  ,[-* - *-]   +  b  E  oC-o-^ —  2 

1  °AH%~k*l        %>^M  VVv 


"2 


+  2b,b9E  9     9[- 

1  ^  e.  .e^.of.ot 


3 


? 
=  -2o?(p-2)b.E         ,[-, *-]   '  2o?(p-2)b  E         ,[   »  .  "       2  3 


3 


il»   1         ^N   ~1  v. 


-3'   2  -M   ^V, 


+  E 


2    _2 


[{- 


"2  -2 

blSN  b2SM  ,2 


]    . 


Sl»S2»°l'°2     Nl&  "   Vvn        M'*M"   Vv. 
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>r   .  „r-l,,r\.r 


sing  the  fact  that  (a+b)r  <  2r_1(ar+br),  for  a>0,  b>0  and  p>l  then 


E  [t_ "A + "A }2]  (4.13) 

h-iiA-i  H*i*-k\  M1Wv2 

"4  S4 

<  2b?E         -[   ,     SN       y]  +  2b*E  [  2.M       ,   ]    • 


Therefore,  using  (4.13) 

extreme  rhs  of  (4.12)  (4-14) 

<   -2o2(p-2)b1E    ,[-5 r]  -  2^(p-2)b  E    ~[  ,.  '\  2  ] 


s4  i.4 

+  2b?E    9l-r-r* H  +  2b2E    2[  2  -  '\  3  ]  " 


~2  V, 


From  the  proof  of  Theorem  2.1  we  have 
i? 


E     [  UL_]  (4.15) 
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0?S2 


=   E     .     E         „[- 


rn 


n=no     Si.^n2!^  -  ^l{     ^M-n] 


?.      ..      .     °? 


n=nQ     ^  n       [N-n]       ^  a*  b^-  Vv 


■  Cn/^rT^n^VV 


■E/VVri  . 


Similarly, 


o2? 


e    of  n   ix..  -  K  k.  °f    v 


(4.161 


o2!2 


-2 

E     p[  2-2M    z-J'EA(Vr]  ' 

~2'   2         ~M       ~2  V0  2 


(4.17) 


and 


^2-4  "4 

°2SM  ,       ..     r_   ,*,    SM- 


E       J-  I-  2  ]  =  EJ>CV  V  TT] 

8      n'lY.-lLr  of      p      " 


&%«*'%' k\        -2 


(4.18) 


where    A    =   A  (Go2)   and    A    =    A  (Go2).     Therefore,   from   (4.15) 
n         n   ~1     1  m         m   ~d     c. 


(4.18; 
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s"2  s* 

rhs  of  (4.14)   =  2bxE  .[g  (^)  jp  (bx  -j  -  (p-2))] 
al  al 


(4.19) 


-2  -2 

SM    ,u      SM 


+  2b2E/9p(Vir(b2  7-  (p"2))]  • 


Since  b,,b2   e  (0,p-2)   then 


"2     "2 
SN   ,SN 


rhs  of   (4.19)   <  2(p-2)b1E   Jgp(  \)  -jp  (-^  -   1)1 


(4.20) 


i2     i2 

+  2(P-2)b2E  2Cgp(y /<4-  1)] 

a2  °2    ■ 


Consequently   using   (4.7),    (4.10),    (4.12),    (4.19)   and   (4.20) 


b,,b 


^i^.^i^2'  -  R(£i.^'iW  (4-21) 


•2      "2 

SN   ,   N 


<  2(p-2)b1Eygp(AN)   fp^-  D] 


i2     I2 


+  2(p-2)b2E^[gp(y  F-(7-'  1)] 


By  applying  the  latter  half  of  the  method  of  proof  for  Theorem  2.1  to 
each  term  in   the  right  hand  side  of   (4.21)    the  result  follows. 
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bl»b2 
4.3  Asymptotic  Risk  Expansion  for  WH^f_and_6|^M 

In  this  section  we  first  obtain  an  asymptotic  risk  expansion  up 

b1,b2 
to  the  second  order  term  for  W^  M  (trivial)  and  then  for   6^M     . 

Observe  that 


Rf 0     9     a2    a2  W        )  (4.22) 

KlSl'S2,al*a2'^,M' 


=  E 


£i»°i  i 

®2»°2  * 

=   a2   tr(Q^)   E  2[N_1]  +  c  E  2[N] 


-1. 


+   <£  trtQVg)   E  2CM     ]  +  c  E  2CM] 
a2  a2 


(a2  trtQV^)         +  c  E     [i^J-] 
°1 


,2 


02 


Following  the  arguments  of  Section   2.3  concerning   the  asymptotic  risk 
expansion  of  X^  we   have  as  c-O, 
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R(2l'£2'al'a2'^N,M) 


(4.23) 


7/    ,  v2     c        v2   2  V2     c 

=  2c       (oj   trtOV^)         +|p  +  2c       (°2   ^(QV^))         +-^+0(0) 


v2 


v2  i/2  v2 

2c       [(°5  trfQVj))         +  (°i  trtQVg))       3  +  j  +  o(c) 


bl'b2 
Next,  we  find  the  asymptotic  risk  expansion  for  ^  M     .     From 

(4.7)   -   (4.10)  and   (4.15)   -   (4.18)   it  follows   that 


2     2     bl'b2 
R(91,e2,a1,a2,5N)M      ) 


(4.24) 


2     2  -  1  SNn 


Jl      ul 


-2         b2  -4 

2(p-2)b1E   2[g   (  y  /]  +  4  E   2[9p(  V  TT] 
al  CT2      a2 


-4. 


-2(p-2)b2E   2Cgp(^)  jjp] 
a2 


2b,b«E  2     2^' 


s*2 


>M 


2l'22'al'a2NS-   V^       M"Lv,-    ^2!lv2 


'   <h  ~   h^llYh^h-   *2,] 
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Now,  for  technical   reasons  we  shall   handle  the  remainder  of  the  risk 
expansion  separately  for  the  four  cases  (i)    ^  -  ^,    ®2  =  ~2' 

(ii)   !i  +  k*   h*  &   (iii)   ~1  "  ~V   %  *  k  and  (iv)   ~1  *  ~1' 
90  =   K.     Since  case   (iii)  and  (iv)  are  essentially  the  same  we 

shall   only  explore  case   (iii).     However,   using  results  of  Section 

2.3  we   have,   respectively,   for    Q.   =    X. ,   1=1,2,   as  c->0, 

1         2  "4  "2 

c"  ^E^V  £)  -  2(p-2)b1Eo2(gp(^)  /)]  (4.25) 

_l/2 
*-   ^1b1(-|7)2(2(p2-4)p"1  -  bx)(tr  Qli)         Kx, 


and 


1         2  -4  "2 

c"  /2[^E^(gp(V  £)   "  2(p-2)b2E^(gp(  V  £)]  (4.26! 

2       2  2 

_l/2 

*"    CT2b2(-pT2)2(2(p2"4)P"1   "  b2)Ur  Q~V  K2* 


where  K,   =    f   4  (1  +  2ta71.f  ^dt.  K,  =   f   4  (1  +  2ta"1)' V2  dt,   and 

the  a..'s,   j=l,...,p,  are  the  diagonal  elements  of  A^.   =  D^.D^,   i=l,2 

(here,   D,.   is  a  matrix  such  that  D^'V.   =  1^  and  D^.d!   =  A^., 
diagonal,   i=l,2).     Also  from  Section   2.3,  we   have,   respectively, 
for    3.    t    X.,   1=1,2,  as  c-»0, 
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and 


.->  fi 


+  -  b1(J7)2(2(p2-4)p"1  -  bjjdjdr  Qli)"1^  -  W 


b2  i4  i 

c_1  ^4E  2(W  TT}  "  2(P"2)b2E  2%(  V  /)] 
°Z     G2  "  °2 


b2(-^)2(2(p2-4)p-1  -  b2)a2(tr  O^)'1  l$  -   ^  lj 


(4.27) 


(4.28) 


Consequently,  we  need  only  concern  ourselves  with  the  last  term  in 

the  right  hand  side  of  (4.24)  for  the   three  cases  we  will   consider, 

However,  we  shall   further  require  use  of  results  form  Section  2.3 
.~  a.s.  ,       ,  a.s.  ,          V2   a.s.       2  Hz 

such  ass*       ^o2,  i2       .  ^o2,  Nc  ♦    (^trCQ^)) 

V2   a.s.       ?  V2 

Mc  *    (o£  trtQVg))       • 

For  case   (i)    ^  =  ^,    %  =  ^  let  ^l  =   ^  (^  "   V  and 
B       *  M  (YM  -    JL).     Then  as  c    *  0,   using  Anscombe's  Theorem, 


5ni 

I      N( 

0 

9 

ih 

0 

B~M2 

-V 

0 

0 

& 

)  , 


(4.29) 


implying 
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~!ni 

I      N( 

0 

, 

%2 

->■ 

0 

alh 


4*2 


(4.30! 


Hence,   from  (4.29)  and   (4.30) 


5*u!l  1    V^  ^M2       L 


l^l"V1    "tM2"V2 


Z     (say) 


(4.31) 


and 


~%lil  9-    ^2  ?M2       L 


"V^    l,B.M2l,2Y2 


*      1   . 


(4.32) 


Since  E|Z|    <   -,   from   (4.31)  and   (4.32),  E[Z]  =  0.     Consequently,   as 

c   +  0, 


\  s 


•"■Vii'vj    MIVk'v2 


-2 

^  IXn-  V'fiV^lM"  ^      (4-33) 


SN 


-2  B^V^^B, 

sM  ~1~1   ~     ~i   ~d 


l/2     l/2  l/2     V2     ,|R      ,2      .a      „2 

(Nc       )         (He       )  'SMl'Vj     ^2"V2 


*  (-^)2 
vp+2 


o  V4        2 

(ol   ^QXl^  (a2   tr{9.l2)} 


v4 
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Next  we  prove   the  uniform  integrability   (in  c  <  CQ)   of  the   left  hand 
side  of   (4.33)   for  case   (i).     Again   using   the  Cauchy-Schwarz 
inequality,  with  probability  one, 

|c"1/2      _**     t     •       .'"     i     (X,  -  i^'fiW'Sl  -   i2>l        <4-34' 
<  c 


N»^-iil,v1     miiIm"Vv2 


Thus  it  suffices  to  prove  the  uniform  integrability   (in  c  <  cQ)  of 
the  right  hand  side  of  (4.34).     With  this  end  note  that 

E  [C-^_J J_ ,    y2     g         ij  ] 


(4.35) 


••4  -4 

-lr  r   -1  SN SM 

<   d     L  ?      ?LC       — *-:  2        2~~ :  2" 


By  previous  arguments 

"4  -4 

c  r   ,     $N M       -    ]  (4.36) 

L  2     2L^„=  ~    .2     M2„7  ~    „2    J 


gp&.of.i  N  ii^-iiiiv1  M  "Im^'v 
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s4  I4 

n  m 


=   S__-    E E  2     2C    2  2        2       ■         2      I[N»nfM=m]] 


n=n     m=n 


0        °£i>£2'ai*a2n   WlV   'SrVv. 


"4     -4 

=   zn=n   Vn  E  2   y~2"X    On,M=m]J  *1*2 
o         o   o     a     an    am 


"4        -4 

Kl  K2  P  rSN    .   SM-, 

77L2     2LTT     TTJ   * 
a     a       or     a 


Then,  using  Holder's  inequality  and  (4.36) 


K1K2    W.Vn,    rV 


lhs  of   (4.35)    <  44  d_iE    '  [-4]   E       C-iiy]    .  (4.37) 

"  a*  o|  a^     C*T       o*     cf-T 


By  following  the  proof  of  Theorem  2.1   for  nQ  >  2,   c  <  cQ  <  1, 

s8  i8 

E     [-4]  and  E  -C-^U]  can  be  bounded  by  a  constant  (depending  on  cQ) 
oj  cN2  cr|  cM2 

uniformly  in  c  <  c   .     Combining   (4.34)   -   (4.37)    the  uniform 
J  -    o 

integrability  of  the  left  hand  side  of  (4.33)   follows.     Thus,  for 
n     >  2,   case   (i),  as  c  +  0, 


V  s2  i2 

c      2E  2     2^"^ r-^T-(XN-X1),V-V1V-1(YM-X2)] 

lvlrava\  Hih'h\  **hrh\ 

(4.38) 
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.  P  ,2 !i 


2  V4        2  V4 

(al   tr(5Il)}  (a2   tr(9l2}) 


ECZ] 


Consequently,   for  nQ  >  2,   case   (i),  as  c  >  0, 


2     2     ^i'^2 
R(Sl'&'ffl*a2'Sl.M     ) 


(4.39) 


V2        o  V2  2  V2 

=  2c       C(oJ   trlQV^)         +   (^   trtQVg))       ] 


V; 


v2 


c       [a1b1(-5r)2(2(p2-4)p_1   -  bx)(tr  QV^  ^ 


-  V2  v2 

+  a2b2(_E2-)2(2(p2-4)P"1  -  b2)(tr  QX,)         K2]  +  o(c       ) 


Thus,   in   this  case, 


2     2  -  2     2       12 

R(91,92,a1,a2,W^jM)    -  R( e^'W^M      ] 


(4.401 


l/2  n     2  2  -1  -V2 

=  c       [a1b1(-^2-)2(2(p2-4)p  X  -  bx)(tr  QVJ         K1 


-  V2  V2 

+  a2b2(^P-2-)2(2(p2-4)p"1  -  b2)(tr  QV^)         Kg]  +  o(c       )    . 
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For  case   (ii )    9,    t   x, ,   £    ^  ~?  usin9   the  add"itl'or1a1    fact  tnat  as 
a.s.  a.s. 


c  ♦  0,   X,.       +     9      Y         *     e      one   gets 
~N  ~1      ~rl  ~<i 


"2  ;2 

c"1       .     "     Z     •       .    "     ;     (|„  "  ijl'VJYVXM  "  i2)  <«•«> 

p.  (J^ i i i l_ 

p+2  V?  V2 

(°?   trtflJd))     "    "Wv     (°2   tr{^2)}     "    'Wv 

l«-lw-li 


•    /9     -    X   )  'v      n      V      ( 9     -    x   ) 


as  c   *  0. 

Next  we  prove  the  uniform  integrability  in  c  <  cQ  of  the  left 
hand  side  of  (4.41)  for  case  (ii).  However,  by  (4.34)  and  (4.35), 
we  need  only  bound, 

"4        -4 
E      o  ?[c~2  ,  "H       2       2  .  M   r  3  (4.42) 


uniformly  in  c  <  c     to  prove  the  uniform  integrability  of  the  left 
hand  side  of  (4.40).     With  this  end  note  that 

-4                       -4 
(4.42)   =   l"       SV    E  ,     oC-o^11 o r^ H        (4-43: 


n=n     m=n     Q     Q     o2   o2  cn2BX  -*   n2     cm2  »Y  -x   l|2 

o         °  ~i«~2»   i'  2  cn     *n  ~1  V,   cm     ha  ~2  V 


I[N=n,M=m]^ 
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•s4         ;4 

o         o  a-.  >(3o  cng.    cma^ 


=  JT~2~E2     2Cgp(AN}   cR"9p(AM)   cM"3 
al    ^2      0i»o? 


Following  the  proof  of  Theorem  2.1  we  have  with  probability  one, 

s4  V  i4  \ 

VV  ctf  ^  L1S'5/(NC    2  )2  and  V  V   C*  "  L2»M/(MC       )2  WherS 
L.  and  L?  are  constants  not  depending  on  c.     Thus  it  suffices  to 

bound,   uniformly  in  c  <  c   , 

••4  -4 

c  r     Sn  Sm ]    .  (4.44) 

2     2  1  i 

at ,  cro  V2    o  V2 


1     *   (Nc       r   (Mc       P 


With  this  end,   using  Holder's  inequality,  we  have 


v2     s"2  v2     i8M 

(4.44)    <   E    '[ *— ]   E  2   [ -"— ]    •  (4.45) 

oi            X/2    d  o?            V2    4 

1     (Mc       )4  2     (Mc       )4 


Using  Holder's  inequality  once  more  with   5  e  (0,1), 

"8  2+6  _2_  .2+5  _J_ 

E   J       '»       ]   <  E  ?[(X2cr2     2  ]2+S  E  .[(.■«)    S  ]2+S  .  (4.46: 


"2 

By  the  backward  martingale  property  of  sn  we  can  bound 


Ill 


2+6 
E  ?[(s[!)    6  ]  by  a  constant  independent  of  c.     Additionally,   for 

CT1 
e  in    (0,1), 

E  2[(N2c)"(2+5)]  (4.47) 


al 


<   (n2c)"(2+5)   P  ,(N<en*)   +   (e2a?  tr(QVJ) 

0  c.  1  — X 

al 


•(2+6) 


(V2)(n   -Dp 
Using  the  result  that  P  2(N<en*)  =  0(c  v"         ),   then  for 

al 

n     >  3  and  c  <  c     <   1, 
o  -  -     o 

E  2[(N2c)"(2+5)]  (4.48! 


9     t9±*\      l/2  (n   -l)p-(2+5)  9o  -{?+x\ 

<  K   (n2)_(2+5)   c  °  +   (e2o?   tr(QYJ)    (2+5)    , 

-      0       0  0  1  — 4- 


where  K  is  a  constant  independent  of  c.  Similarly, 

E  ?[ ]  can  be  bounded  by  a  constant  independent  of  c. 

al    2/2  4 

1  (mc     r 

Combining  (4.42)  -  (4.48)  for  nQ  >  3,  the  left  hand  side  of  (4.41) 
is  uniformly  integrable  in  c  <  cQ.  Consequently,  for  nQ  >  3,  in  case 
( i i ) ,  as  c  *  0 , 
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c'h 


o2.^ 


i.^VVVv/'VV'v 


M    r  (VV'iiVV'VV  (4-49) 


<p?/   ai  'Wv*   V%TVh\  (tr^JtrtQV^)) 


-v2 


•  (VV'v^~lv?1(3>~V  • 


Hence,  using   (4.27)  -  (4.28)  and  (4.49)  we  have  for  nQ  >  3,  case 
(ii ),  as  c  >  0, 


2     2     bl'b2 


(4.50) 


V?  V?  V2 

=  2c       [(of*  trtQVp)        +  (oj  tr(QV2))       ]  +  ■£■ 


-Ll2(?(n2-4)n"1-h.)a? 


-l.a         1       „"2 


-  cb1(-^-)£(2(p^-4)p  "-b^ojttr  QV^      ""Sril'v 


"  cb9(-g7)2(2(p2-4)p'1-b?)o?(tr  QVo)"1"^-^" 


1  11  q    _  1     n-^ 


2vp+2 


2'   2V       —2'       ~2   ~2  V. 


P  .2 


2cbib2W  VSrVvi  yvVv^^i^SV* 


-v2 


^-^'fiV^Sr^  +0(c) 


Thus,   in  this  case 
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R<2i'^VCT2'W  "  r(2i^'V02'^1m  2)  {4-51) 


+  b2(2£T"-b2)    4(trQi2rl    'V^ 

_l/2 

+  2b1b2   cr1nei-X1^2a2ii02-X2iiy2{tr(QV/1)tr(QV2)} 


•  ^V^Y^V^3  +  0(c)  • 


For  case   (iii)   e,   =  X,,   e^  /  Ag,  using  Anscombe's  Theorem,  as 
c  +  0, 

L  9 

BM1      +  N(0,afvj                                                                 (4.52) 

~N  1  ~     1  ~i 

and  hence 

L  «, 

-BU1      ♦  N(0,afY1)                                                                    (4.53) 

~N  1  ~     1  ~1 


Consequently,   using   (4.52)   -   (4.53) 


2N1  L 

*      Z     (say)  (4.54) 


2 


and 
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ail       l 

*      Z   .  (4.55) 


|8JU"Y 


Here   (4.54)  and   (4.55)   imply  that  E[Z]  =  0.     Consequently,   using 
previous  results  and   (4.54),  as  c   +  0, 

-2  -2 

c-3/4       .  S"  "     z     (XJ1-X1)'VJV1VJ1(Y>X2)  (4.56) 

2  a2 

lp+2      9  1/4      ,  V2  ~u  -  ^  ~*  a 

(ffj  trtQVj))  (a*  trtQV^)) 


Next  we  prove  the  uniform  integrability  in  c  <  cQ  of  the  left  hand 
side  of  (4.56)  for  case  (iii).  By  (4.34)  and  (4.35)  to  prove  the 
uniform  integrability  in  c  <  cq  of  the  left  hand  side  of  (4.56)  we 
need  only  bound, 


"4  r4 

r  r-m       sn m       1  (4  5r 


uniformly  in  c  <  c   .     Using   (4.36)  and   (4.43)  we  have 

"4  -4 

,4.57)  =ijVZ     A^VVl1'  <4'58) 

°1   "2     V°2         „     * 

Nc 
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By  previous  arguments  we   have 

K     L  s"4         i4 

rhs  of   (4.58)   l44E   2     2  C— \ T-3   "  (4>59) 

a-,    a0      a, ,  a9  V2      M  c 

1     ^        x      c  Nc 


Following  previous  arguments,   for  c  <  c     <  1  and  nQ  >  3,   rhs  of 
(4.59)  can  be  bounded  by  a  constant  independent  of  c.     Combining 
(4.57)   -   (4.59)   the  uniform  integrability  of  the   left  hand  side  of 
(4.56)  follows.     Hence,   for  n     >  3,  case   (iii),  as  c  +  0, 


Qx>02>°ro2  NH^-X1Hvi  M|l!^|-^2ilV2 

(4.50) 
2  2 

*  ^2  — - — 5" — - — irz  aiy^l$^] 

(cti  tr(5!i)}      (a2  tr(5!2)} 


=  0. 

Consequently,  using  (4.25),  (4.28)  and  (4.60),  for  nQ  >  3,  case 
(iii ),  as  c  +  0, 

*(^,44>knZ)  (4-61) 

V2   2      V2    2      V2 

=  2c   [(oj  tr(QV:))    +  {a2   tr(Q\y)   ] 
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1/  -  V?  V2 

c    2a1b1(-|7)2(2(p2-4)p"1-b1)(tr  QV x )         Kj_  +  o(c       ) 


Thus,   in   this  case, 


2     2 


2     2     bl'b2, 


^I'^'VVW   "  R(2l'£2'al'a2»^,M     } 


(4.62) 


V?  o  2  ,  -V2  V2 

.  c       ^(p^)     b1(2£-p--b1)(tr  QVj)         K1+o(c       ) 


For  case   (iv)   9.   *  x. ,   0O  =  x_,  following  the  arguments  of  case 
(iii),  we   have   for  n     >  3,  as  c   ♦  0, 


2     2    rbl'b2. 


R(®l'®2'al»CT2»iN,M     } 


(4.63) 


V2        ,  V2 


v2 


=  2c       [(oj  trtQVj))         +  (<£  trtQVg))       ] 


^  >■      ,2  ,..,_ 2      ,,     "I 


v2         v2 


c       a  b   t^2-r(2(p^4)p"i-b2)(tr  QV2)         K2  +  o(c       ) 


Thus,   in  this  case 
2     2 


2     2   A'b2. 


R(91,92,a1,a2,WNjM)    -  R^.e^.a.,,  6^      ) 


(4.64] 


v2 

c       a 


-  v2         v2 

(_P_)2  b0(2(p2-4)p"1-b9)(tr  QVJ         K„  +  o(c       ) 


2vp+2'       2 


2'         "2 
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Remarks 

For  case   (i),   the   leading  term  in   the  risk  difference 

R(2i'22*°1'°2'Hm,M)   "  R(®l'®2'al'02'~N!M  2)   iS  maximized  at 

b     =  b     =  p-2  when  we  restrict  ourselves  to  the  intervals   (0,p-2]  for 

values  of  b.   and  b„.     Accordingly,   the  asymptotic  risk  expansion 

suggests  the  optimal   b,   =  b2  =  p-2.     This  is  different  from  what  we 

have  seen  in  Chapter  2,   since  there  we  allowed  the  interval 

(0,2(p-2))   for  b,  and  accordingly,   the  optimal   b  turned  out  to  be 

(p2-4)/p.     For  case   (iii),   the  optimal   choice  of  b1  in  the  interval 

(0,p-2]  turns  out  to  be  bl  =  p-2,  although  we  cannot  say  anything 

about  the  optimal   choice  of  br     Case   (iv)   is  just  the  opposite  where 

the  optimal   b2   (in  the  interval    (0,p-2])   turns  out  to  be  p-2,  but 

nothing  can  be  said  about  the  optimal   by     The  optimal   choice  of  b1 

and  b2  in  case   (ii)   seems  to  be  the  most  unclear,   since  the  leading 

2     2  2     2     bl'b2 

term  in   the  risk  difference  R(91.02.o1.a2»ilN,M)"R(~l,~2,oro2,~M,M     ) 

does  not  reveal   any  optimal  b.  and  b?  free  from  the  unknown 
parameters. 


CHAPTER  FIVE 

SEQUENTIAL  SHRINKAGE  ESTIMATION  OF  INDEPENDENT 

NORMAL  MEANS  WITH  UNKNOWN  VARIANCES 


5.1  Introduction 

Let  X, ,  X?,  .  .  .  be  a  sequence  of  independent  and  identically 

distributed  N(0,z)  variables  where  9eRp  is  unknown  and  Z   = 
diag(<^,...,a2).  The  problem  is  estimation  of  9.  Based  on 
X  ,  .  .  .  ,  X  ,  1f  6  =5  (X1,...,Xn)  is  used  to  estimate  0,  suppose 
the  loss  incurred  is  given  by 

L(9,6  )  =  (6  -  9)'  Q  (5n  -  9)  +  en,  (5.1) 

where  Q  =  diagtq^...  ,q   )   is  known  with  q^  >  0,   j=l,...,p  and 

c(>0)  is  the  known  cost  per  sampling  unit.     Stated  differently,   the 

problem  could  be  thought  of  as  simultaneously  estimating  p 

independent  normal  means  with  unequal   variances,  where  we  sample  an 

equal   number  of  times  from  each  population.     For  a  fixed  sample  of 

size  n,   the  most  natural   estimator  of  9  is   the   sample  mean  vector 

X     =  —  l"     X.,  with  risk 
~n       n     i=l~i 


R(0,z,X   )   =  -  tr(QE)   +  en    .  (5.2) 
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v2 

If  I  is  known  the  above  risk  is  minimized  at  n  =  n*   =  (tr  Q£/c) 
Note  here  that  tr(QE)  =  E^P/j-     However,  if  £  is  unknown,   there 
does  not  exist  any  fixed  sample  size  which  minimizes   (5.2) 
simultaneously  for  all    a2,   .   .    .   ,   <y     Motivated  by  the  definition 
of  n*  we  proposed  the  following  sequential   sampling  scheme. 

v2 

N  =  inf{n  >  m  :  n  >   (tr  QWn/c)       }   ,  (5.3) 


where  m(>2)  denotes  the  initial   sample  size,  Wn  =  diag(sn],,. ..  ,snp) 
and  for  j=l,...,p, 

s2    =    1    ZJ     (x      _R     )2  ,  (5.4] 

nj       n-1     k=l     kj         nj 

for  every  n  >  2  and  Xpj  s  j^  z[si\y     The  above  stopping  rule  was 
first  proposed  by  Ghosh  et  al .    (1976)  and  Woodroofe   (1977)   for  the 
more  general   case  of  an  unknown  positive  definite  £. 

To  show  that  the  above  rule   (5.3)   can  be  motivated  from  a 
minimax  criterion  we  need  only  follow  the  arguments  of  Section  1.1 
with  a2V  replaced  by  I  and  or  B,   the  prior  covariance  matrix  of  9, 
replaced  by  Z  =  diagtz^. . .  ,z   ).     Then  the  Bayes  estimator  of  9  is 


£nLV  ^^ip^L2"1)"1^-^  • 


with  corresponding  posterior  risk  equal    to  —  trtQ£(/Jp+  n* ~     V      ■'+Cn' 
Again,    the  posterior  risk   is   independent  of  any  unknowns   so   that  the 
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Bayes  sequential   decision  rule  is  a  fixed  sample  rule  with  sample 
size  determined  by  minimizing  -  tr[QE(I     +  —  Z     £)      ]+  en  with 
respect  to  n. 

Next  consider  the  sequence  of  {N(u,m_I   ),  m>l}   priors  on  9. 
Then  the  corresponding  sequence  of  Bayes  estimators  is 

6B(X   )   =  |i  +   (I     +^)_1(Xn  -  ii ) 
^n  ~n        ii        ~p       nm~        ~n 


with  the   sequence  of  risks  being  -J-  tr[Q£(I     +f^£)      1  +  ^-     Since 

-  trCQzd     +  —  z)"1]  +  en  ♦  -J-  tr(Qz)  +  en  =  K  as  m  ♦  »  and 

Rte.E.X   )  =  K  for  all   9  e  RP  then  n*.   the  sample  size,  and  Xn>   the 
estimator,   is  a  sequential   minimax  rule.     Consequently,  we  can 
consider   (5.3)  as  an  empirical  minimax  stopping  rule. 

In  Section  5.2  we  develop  an  asymptotic  risk  expansion  up  to  the 
second  order   term  for  L  and  for  a  class  of  rivial   estimators 

£n(£i",,,£n)'  where 

""in  *'w 


for  every  n  >  2,    |X  -xiS  =  (X  -a) 'w'VV^X  -x) ,  b  is  a  constant 


an 


d  x  e  RP  is  the  known  point  towards  which  we  want  to  shrink. 
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5.2     Asymptotic  Risk  Expansion   for  XM  and   5f| 


First  we  obtain  an  asymptotic  (as  c   +  0)   risk  expansion  for  the 


risk  of  L  and  then  for   fijj.     Observe  that 


^^bh]  -  Ee,E[(^),Q~(^-2)]  +  cEE[N]  (5'6) 

=  tr(QE)EJN_1]  +  cE  _[N] 


Atr^c^i^] 


a.s. 


a-s-  n  2  p  2 

From  definition  N       >    <°  as  c  +  0  so  that  i^q^s^     +     ^^j0]- 

Also,   using  the  Helmert  orthogonal   transformation  X^  = 

(X,  .+  ...  +  Xu  .    ,  -  (k-l)X.  .)//k(k-l),   j»l.....p  and  k=2,...,n, 


write 


tn^)  ■  «5-ivJj  (5-7) 


'nMW^ 


1     vn     r  rp     _  v2  -, 


Hence  tr(QW  )   is  the  mean  of  (n-1)   independent  and  identically 
distributed  random  variables  with  mean  tr(Q£)   =   S^  ^q^cr  and 
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rP       2  4 


variance  2tr(Q 2&)  =  2 sR=  q  a  .     Therefore,  using  Anscombe's  Theorem 


/N"(tr(QW,)  -  tr(QI))    +  N(0,2tr(Q2£2))   so  that  using  the  delta  method, 

~~N  ~~  ~  ~ 

V?  V?      L  o   o 

v¥((tr  QWN)         -  (tr  QI)       )   -  N(0,  V2  (tr  (fl  )/(tr  QI) ) . 

Next  use  the  inequality 

V2  1;2 

(tr  QW  /c)         <  N  <  m  +  (tr  QW^/c)         •  <5-8) 


Dividing  all    sides  of  (5.8)   by  n*,  making  c  +  0,  and  using 

V?a.s.  \  a-s- 

(tr  QW   )  *     (tr  QI)        ,   it  follows   that  N/n*       *     1  as  c  +  0. 

V?  V2     L  o  2 

Thus  /n*((tr  QWN)         -  (tr  QI)       )  -  N(0,V2(tr  Q  I  )/(tr  Ql)). 

Again  from   (5.8), 


V2  v2  v2 

fF  ((tr  QWN)         -  (tr  QI)       )/(tr  QI) 


(5.9) 


_  v2  v2  v2 

<  (M-n*)//rF  <  m//n*"  +  /n*"  ((tr  QW^)        -  (tr  QI)       )/(tr  QI)       . 


Hence  from  (5.9),  as  c  ♦  0,   (N-n*)//fi*"  +  N(0,  V2(tr  Q2£2)/(tr  QI)    ), 
implying  that  (N-n*)//fT*  N(0,  V2(tr  Q2£2)/(tr  QI)2)  as  c  *  0  since 
N/n*     V    1  as  c  ♦  0.     Also,    the   uniform  integrabili ty  of   (N-n*)   /N 

for  all   c  <  c     (specified)   can  be  proved  by  repeating  essentially  the 

2 
arguments  of  Ghosh  and  Mukhopadhyay   (1980).     Thus  E£C(N-n*)   /N]  * 

(  l/2  (tr  p2£2)/(tr  5£)2)   asc+0,   and  thus,   from   (5.6),  as  c  ♦  0, 
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V?  lll  o  o  2 

R(e,Z,XM)=2c       (tr  Q£)         +  f  ( tr  Q  f)/(  tr  Q£)     +  o(c)   .     (5.10! 


^' 


Next  we  develop  the  asymptotic  risk  expansion  of    fijj 
note  that 


.     First 


R(9,£,^)  =  R(£.£»^) 


(5.11) 


2bE9E[     -1       2  (^V(*Jrg)] 

H»i  n  ixm-xil, 

~N    ~  W 


+  b2EQ  I[-T-^ 2] 


Denote  by  B    .  the    a-algebra  generated  by  Y.  .,  i=2,. 
n  j  'J 


,n,   j=l,...,p, 


and  Xni,   i*j=l,...,p.     Then 


~  ~Hih  ~"w 


(5.12) 


~  ~nlV~"w 


,J1    ~  w 


LWlflf^fc?  <  W  ^j'W  WV 


n  uX  -xii,, 
~n   ~  « 


124 


■  ^^e./'CN^.r^^  VV^'VV'V1  ■ 


where  in  the  last  equality  in   (5.12)  we  use  the  fact  that  I[N=n]  is 

B      measurable   i=l,...,p.     By  Stein's  identity  and  the  mutual 
nj 

independence  Xnl>    .   .   .   ,  X      and  W^,   for  j=l,...,p, 


h,^-rh7(VVsn1(VV'W  (5-13) 


Hn'!n-i«H 


A     a      !VV!nj   |B     , 
e,rn     s       nilv  _x.2        nJ 
3  nj        "in   ~  M 


iE       [(     >•       -2         SnJ       (X"JV    )|b     ] 
0       ~~    ni^-xnj  niX^-xiy      qjSnj 


Using  (5.12)  and  (5.13), 


R(e.s.«tj)  =  R(e,z,L)  -  2bE  [-^ — ?  £=1  -f-]       (5.14) 


N  ■&-i,H     SNj 


4hF    r     i      j   !VV!i_- 

+  4bEe  /""F^ 4  *5-l 4 r- 


+  ^G  J2-1   2] 
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For  technical   reasons  the  remainder  of  the  asymptotic  risk  expansion 
will   be  broken  up  into  the   two  cases   £/    A  and    9=    X. 

For    e  ;*    X  denote  by  B     the    o-algebra  generated  by  Y... 
**      ~  n  'J 

i=2,...,n,   j-1 p.     Then  realizing  that  I[N=n]  is  Bn  measurable, 

E       r         1 ]  (5.15) 


~J1    ~  w 


■  Cm  TEe.#  ftfta^'DHO  |B"] 

'■Ml     ~  n 


Zn=m7rEe,E[I[N=n3Ee,E   [    ■,        21  Bn }] 

~  ~    n  "in    ~"w 


From  Cressie  et  al.    (1981)  and  the   independence  of  X^  and  Y„, 
i=2,...,n,   j=l,...,p,  we   have 

W"^— 7  I   V  (5<16; 

-A  ~  w 


f  Ee  jlexpt-tni^-xij)   |  Bn]dt 
0     ~'  ~ 


=  rE8,^PHn^^-)    IBnMt 
u     ~  ~  Mj  nj 
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•C1?    Ee,ECsXp(-tnV^-)    I3n]dt» 


j=l 


q  .s 


where  in  the  last  equality  in   (5.16)  we  use   the  mutual    independence 

of  the  X     's,   j=l,...,p.     Using  again  the  mgf's  of  chisquared  random 
nj 

variables  for  each  j=l,...,p, 


h,^{^-f^~]    |Bn] 
Mj  nj 


(5.17) 


v2 


( 4  j  "J     2)     '   exp(-tn  -~j 


(ei-xi)c 


qjSnj  +  2t°j 


q  .s    .  +  2ta- 


since  "(X    .-x.)2  ~  x2( i-J— ).     Combining   (5.15)   -   (5.17),  we 

2     nj     j  *1         ?  * 

°j  2aJ 

have 


E_     [ " 


(5.18) 


i  q  .su .  V?  n  (6i"Xi) 

~         °     j-1  qjSNj       2toj 


'j=1  "jsNj  +  2S 


Next,  note  that 


-1  1 


Q,-s. 


..    _    x     ,»     ,   £   ,  'j    Nj 


0    j=lVNj  +  2tCTJ 


j)       }  exp(-tN  e?,i r"^ 2 


(VV 


r)dt     (5.19) 


'JJNj 
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0 


P  ,   qJSNJ 


j=1  q.sj.+2t^c 


\ 


V2  n   (9rx-;) 

exp(-t(Nc  )tP.=1 i-^— )dt 

4     72  2 


.  a.s.  .      V2  a.s.      V2 

As  c  ♦  0,  sj.   +  0.   and  Nc     ♦  (tr  Q£)   .  Hence,  as  c  *  0, 


rhs  of  (5.19) 


(5.20) 


a.s. 


-V2  V2    n    (VV 

(tr  QI)    /  exp(-t  (tr  Qf)    Ep.  ,   J  J4  )dt 
0  J     Yd 


(e.-x.r 

=  (tr  Q£)  (^.i— M— ) 
J    q  .a . 


Next  we  prove  the  uniform  integrability  in  c  <  cQ  of  the  left  hand 
side  of  (5.19).     With  this  end,   note  that, 


lhs  of   (5.19) 


(5.21) 


(0.-X.T 
<-^r{/1exp(-tNlP  I     J       2)dt 


(e.-x,)' 


q  .s.. . 
P   ,         J  NJ 


*  exp(-NiP=1       -j     J  ^^r    f  (;■  4     ,„.  2- 


1/2      1 
)      dt 


VnJ  +  2°j      X    J-l  qJSMj  +  2t*j 


.1      il/rP  (9j"XJ}      T1   +    1    (£P  (9j"Xj) 


K 
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1<J5P      NJ  1  j=i    »'j 


?     7  K  2     P+2 

<     -        max     (s',)     +-5-      max     (sNir 

NZc     Kj<p       NJ  hTc     l<j<p         J 


where  K(>0)   is  a  generic  constant  not  depending  on  c.     Thus  it 

2-1  2     2 

suffices   to  prove   the   uniform  integrability  of  (N  c)       max     (sM.)    , 

l<j<p   NJ 

and  (N2c)_1  max  (s2  )p+2  separately.  We  shall  prove  in  general 

l<j<p   NJ 
the  uniform  integrability  of  (N2c)_1  max  (sN.)  ,  k  >  2  which  will 


KJ<P       Nj' 


handle  all   of  the  above  terms. 


E.[(N2c)-1  max     (s2.)k   I       ,       ,  2     k         1  (5.22) 

I  Kj<p       Nj         C(rc)   1  max     (s^)K>d] 

1<J<P         J 


<  d_6E .[{(N^cf1  max     (s')K}i+0]    , 
I  l<j<p         J 


where  k  >  2  and   6  e  (O,1/^).     Taking   fi'e  (6,  V4  ) ,   and  using  Holder's 
inequality,  one  gets 


2    v-1  /.2    Nkll+51  re    or,) 

r)        max      1  s..  . )     h  I  kd.cji 


E_[{(N'c)_i  max     (sj.)*}1   °] 
~  Kj<P       NJ 

(1  +  5'  )  (1+6) 
<  [Ey(N2cr<1+5,)](1+6)/(1+6X(  max     s2)  &'~&       ^'^W) 

'     I  ~  i<j<p     J 
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(  V2  )m 
Following  Ghosh  et  al.    (1976)   one  gets  Pz(N<en*)   =  0(c  ),   for 

0  <  e  <  1.     Consequently,   following  the  arguments  of  the  proof  of 
Theorem  2.1,  E  [(N2c)~tl+fi  ']  can  be  bounded  by  a  constant 

independent  of  c  for  m  >  3  and  c  <  c     <  1.     To  bound 

E  [(  max     Sy.)r],  r  >  1,  we  need  the  following  two  lemmas. 
£    l<j<p       J 

Lemma  5.1     Let  {U    .,F   }  be  a  backward  submartingale,   j=l,...,p. 
l  nj     nJ 

Then   {  max     (U    .),F   }  is  a  backward  submartingale. 
l<j<p       nj       n 

Proof:     Note  that 

E[  max     U    .|F„x1]   >     max     E[U.|Fn+1]  (5.24) 


^^p^'^-^p^1^11 


>     max     U 
Kj<p     n+l,j 


Lemma   5.2     Let  |U   ,F   }  be  a  backward  submartingale.     Then   {U   ,F   j.r^l, 
L  n     n  n     ii 

is  a  backward  submartingale. 

Proof:     Note  that,  by  Jensen's  inequality, 

E^|Fn+1]>(E[Un|Fn+1])r  (5.25) 


>  U'        . 
-     n+1 


2     r 

Therefore,   by  Lemmas  5.1  and  5.2,    (  max     s    .)    ,   n  >  m,   is  a 

l<j<p     nj 
p 
backward  submartingale  since  s    .,    j=l,...,p  is  a   backward 

martingale.     Hence,   by  Doob's  maximal    inequality  for   submartingales, 
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Ej(  max  s2,,.)r]  <  E,[sup  (  max  s2)'']  (5.26) 


1  Kj<p  NJ    "  £n>m  l<j<p  nj' 


<  K.E_[(  max  s2  )r]  , 
~  1<J<P 


where  K.  is  a  constant  which  may  depend  on  r,  m  and  Z,   but  not  on 

c.  Combining  (5.22)  -  (5.23)  and  (5.26)  the  uniform  integrability 

(in  c  <  c  )  of  (N2c)_1  max  (s2)k,  k  >  2,  follows.  Thus  for 
"  °         l<j<p   NJ 

m  >  3,  c  <  c  <  1  the  uniform  integrability  of  the  left  hand  side 
of  (5.19)  follows.  Consequently,  as  c  +  0,  for  m  >  3  and 
0  *  X, 


^A  r   {  |  (   4jSNJ   2-)  VZ  }exp(-tN^=1  _^V   )dt]  (5.27) 

,      (0.-X.)2   . 
♦  (tr  QS)"^^-!   J  4  )   • 
qjaj 


Next,  observe  that  for  9  /  X,  we  have 


2 

E       [ I eP.      -^-]  (5.28) 

£*£  N2bXm-xb5     J       s2 

~N   ~  W  Nj 
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Zn=oiEe,ECn2 "J         2  Zj=l   s2_  X[N=n]J 
~  ~  n   "in  ~  W  snj 


i  * 

CmE0,zEe,EC TT=        2  Ej=l  "T  xCH-n]lBn3 

~n  ~  w  nj 


■  C/oA  *U  i~  hn^eJ-rbj  '  Bn'] 


nj 


~  ~  n«V*'w 


q,-s^    .  M> 


(e,-x,)' 


=  E  [1  iP     -Lfji  (      jNj     n)       }exp(-tNEP  j    J     y) 


dt] 


Here,  as  c  +  0, 

-1   1   _p         J      r    r   P   i     HJ  Nj 


(e.-X.T 

P       i—J )dt 


¥  Ji-1  4-  i     I  f  (— P4'       }exp(-tNE'  j    ;    2 


(5.29) 


a.s.  -V2     „  V2  O.-X   )' 

+  p(tr  Ql)  /     exp{-t(tr  Ql)         Zv.,       J     i 

0  "  J-         qdad 


[dt 


-l  d    (vV  -i 

=  p(tr  Qe)   1(ZP^1       J     I     )        , 


Vj 


a.s.  4    a.s. 

since  as  c  +  0,   sjj.       ♦     a^,  sNj       >     a..,   3=1,. ...P  and 
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V?   a.S.  llz 

Nc  ♦  (tr  QE)       .     Next  we  prove  the  uniform  integrability 

(in  c  <  c  )  of  the  left  hand  side  of  (5.29).     By  previous  results 
Ihs  of   (5.29)  (5-30) 

<lP      4*4"    -x     (s^)2+-^    .ax     (s2/+2] 
"     J=1  s2     N2c  Kj<p       Nj  N2c  Kj<p       NJ 

Nj 

<   (sR  .   a2)     max     (s:?)[4-    max     (s2)2  +4"    maX     (sn/+2]   ' 
"       J=1     J     Kj<p       NJ     N  c  l<j<p         J  Nc   l<j<p         J 


Thus  it  suffices  to  prove  the  uniform  integrability  of 

max     (s~2)(N2c)_1  max(s2   )k,  k  >  2,   in  c  <  c  .     First  we  require 
l<j<p         J  J 

the  following  lemma. 

Lemma  5.3     Let  {U    .,Fl  be  a  backward  martingale  and  let  U    •  >  0,   for 
— — —— —  i   nj     n  "v 

each  j=l,...,p.     Then   {  max     U~1»F  }  is  a  backward  submartingale. 
l<j<p     nj 

Proof:     Note  that  f(x)   =  —  is  a  convex  function   for  x  >  0. 
Therefore,  using  Jensen's  inequality  for  conditional   expectations,  we 
have 

,-1.-       -i  .     _..  rr.rli 


«,«    %]K^  >-    «p «UnjIFntl] 


(5.31) 


>    max     {ECU     IF^]}"1 


max     U"1 


l<j<p     n+l,j 
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By  Lemma  5.3     max  (s    .),  n  >  m,   is  a  backward  submartingale 

l<j<p     nj 
2 
since  s    .,   j=l,...,p,   is  a  backward  martingale.     Now,   for 


6  e   (0,  V4), 


E-[  max     (s"^)(N2c)_1(   max     s„.)k  (5.32) 


Tuto1-** ijp  Nj' 


[  max  (s^M^cf1!  max     s2  )k>d] 
Kj<p       J  Kj<P       J 


<  d'5Ey[{  max  (s~2)  (N2c)_1(  max     s2   )k}1+6] 
£     Kj<p     NJ  l<j<P     NJ 


Using  Holder's  inequality  for   5' e  (6,  V4  ) , 

Ey[{  max   (s"2)(N2c)_1(  max     s2)k}1+6]  (5.33) 

£     Kj<p     Nj  Kj<p     Nj 


<   [Er{  max   (sM.)(N  c)      }         J 
~  Kj<P     NJ 


(l+fi')(l+«) 


rc    |  mv      c2    ,k  6^6  1(6'-6)/(l+6') 

[E    {  max     sN.}  ] 

~  l<j<p     NJ 


Using  Holder's  inequality  again,  one  gets 

E_[{  max   (s"2)(N2c)"1}1+6']  (5.34) 

~     l<j<p     NJ 

<[EE{max(s-2)}2+26']1/2[E,(N2c)-(2+26,)]    \ 
~       £l<j<p     Nj  I 
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By  Lemma  5.2  and  Doob's  maximal   inequality  for  submartingales 

E_{  max   (s^)}2+26'    <  E    {   sup     max   (s~2) 
t  Kj<p     NJ  Z     n>m  l<j<p       J 


2,, 2+25'    .  r    r   ....     _„   „-2|i2+2«'  (5>35) 


„   _    r  -2i2+26' 

<  K  E   {  max     s    .} 

i  ~  Kj<p     mj 


<nP     E   [s-2(2+2«')] 


where  L  is  a  constant  which  may  depend  on  m,   <5  and   l  but  not  on  c. 
Here,  for  j=l,...,p  and  m  >  6, 


EprJ(««')]  (5.36) 

2    mj 


r   2,,      in-(2+26')   F   r,    2     .-(2+25)-. 


m-1     m-1     , 
(  V,  )  2    x  2        e'*'2 

.[Ami]-'"',T»-,w'1- fi <* 


-  [a2/(n-i)]-(2+26,)(V2)2+26,  r(i!£-  (2+2«'))/r(Si) 


j 


<  °°  , 


2  v"  "     '"*'    2 


since   6'e   (S,1^)  and  hence  ^  -   (2+26')   >  0.     Following   the  argu- 
ments of  the  proof  of  Theorem  2.1,  using  the  fact  that  P(N<en*)   = 

(  V2  )m  2   .-(2+26' ) 

0(c  )   for  0<e<l  and   that  m  >  6  we   have  Ej,(N  c) 
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bounded  by  a  constant  independent  of  c  for  c  <  cQ  <  1.  Also,  by 

previous  results  E  [  max  si?./,  r  >  1,  can  be  bounded  by  a  constant 

£  l<j<p   J 
independent  of  c.  Combining  (5.32)  -  (5.36),  the  uniform 

integrability  in  c  <  c  of  the  left  hand  side  of  (5.29)  follows 
for  m  >  6  and  c  <  c  <  1.  Consequently,  for  m  >  6  and 

9  /  X,  as  c  +0, 


c-4 .d  A_,  3-  /"  ( *  (    y*1  /'}  <5-"> 


•  rj-  rp  _j_  r  ;  ii  ( ±Ji* — _ 


(0,-x,)2 


.  exp(-tNSP  . ji-J — jr)dt] 


,2 


p  (tr  Qr)"1(gP.1   J  J4  )   • 


Next,  observe  that  for  e  t  \, 


2 
E   [  ,  1   ?  (  min  3-)]  (5-38) 


<  c      r   i    yP    NJ J .  -J_i 

"   ^VnX  ->„4   j=1     Q  S4      S2 
~  -  N  "5ji"w        qjSNj    SNj 
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<     E„  „[• 


(  max    -4-)]   . 


»£  N2nXN-Xii^     l<j<p  s 


Nj 


By  previous  results, 


Ert  „[■ 


(  max    -5-)] 


(5.39) 


»  E  A  <  max    -4-)   f  {  £  (" 


q  .s.. . 


"ZUN 


l<J<PSNj        0     j=1   q.sH.   ♦  2ta. 


r)       } 


orx.r 

exp(-tNZP.    .  ji— ^ y)dt]    . 

J_i  q.s„.  +  210*: 
MJ  Nj  j 


a .     a.s, 


Here,  as  c  +  0,     max    -4-      +     1.     Hence,  as  c  +  0, 


1<J*<P  s 


Nj 


^l^T-nf'./lJ   j 


•) 

l<J<P^j'°    "^  qjsjjj  +  2toJ 

,2 


.  exp(-tNzp.  .  j^-^ xOdt 


(5.40) 


a.s.  -V2  V2     _      (e.-x.P 

*     (tr  Qz)         'I*   /"  exp{-t(tr  Qz)         zK.       J     j     }dt 
0  -  ^       q   ffj 
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(e.-X.)2     , 
(tr  Q£)~1(^=1       J     I     r1 

q  .a . 


Also,  by  previous  results,   the  left  hand  side  of  (5.40)   has  been 
shown  to  be  uniformly  integrable  in  c  <  c  ,   for  m  >  6  and  9  ^  X. 
Consequently,   combining   (5.39)  -  (5.40)  and  the  uniform  integrability 
of  the  left  hand  side  of  (5.40),  as  c  >  0,   for  9  /  \  and  m  >  6, 


1 

c_1Efl  -[-*—* j  (  max    4-)]  (5.41) 

~  ~  N    "!n~*"w     X-j-P  SNj 


,  (e.-x.)2 

(tr  Q£)-1(^=1      J     j     )_1 
J  q  .a . 


By  previous  results, 

2 

Es  y^-T-^ 5"  (  min  ~?")]  (5'42) 

~  ~  N  "V-'w  ^j-p  SNj 


2  4  y 

^N      lSKPiJj        0     j=1   qjsjj   ♦  2ta. 


(0.-X.)2 

•  exp(-tNzp.  .  ^-^ T)dt]   . 

J       qsZ-  +  zta. 
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a .     a.s. 


Here,  as  c  +  0,  min  -4-   +  1.  Hence,  as  c  *  0, 
1<J<P  sNj 


^    .«  .  „        q,-s,,,        V2 


c"l   1{  min     _!_,/-  {  £  (  *"'       2)        }  (5.43) 

1<3<P   sjjj     0     j=1  qjS^  +  2t0j 

(e.-x.)2 
•  exp(-tNJ:P=1 4J     J^  ?)dt 


-   )2 
*■    (tr  Q£)-1(^=1   V   j   j     )    • 


a.s.  ,     .       (Q,-^-)2 


q  .a, 
J   J 


Note  that  with  probability  one, 

c-l  L  min  °-Ur   {  l   (  YHi       2)  ^  1  (5.44) 

(e.-x.)2 

•  exp(-tNzP  j^-^ o)dt 

J_i  q.sj.  +  2ta. 
MJ  Nj     j 

2  4     i, 

a.  q  .s..  .     7? 

u"1^,  4~)f{f(   4J  J   2>   J 
1<J<P  tjj  0  j=1  qjSJ .  ♦  2ta. 

(e,-xj2 


•  exp(-tNzP  : 4^-^ 2)dt  ' 


r~x   q.s„.  +  2ta'. 
MJ  Nj      j 


where   the  right  hand  side  of   (5.44)   has  been   shown   to  be   uniformly 
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integrable   in  c  <  c     for  m  >  6  and   9  /   X.     Hence   the   left  hand   side 
of  (5.44)   is  uniformly  integrable  in  c  <  cQ  for  m  >  6  and  e  t  x  so 
that  as  c  *  0, 


i       i  Ji      =0      d  qisNi  1/2 


-lr[l(1n    ^-)f{^(      ?'NJ      »>"l  {5'45) 


o      (erxi)2 

exp(-tNzp.  ,  r^-^ 2)dt] 


i    n      (e,-x.)2 
>  (tr  Qz)_i(zp.  .       J     I     )   . 


Combining   (5.38),    (5.41)  and   (5.45),   as  c  +  0,   for  9  /  X  and  m  >  6, 


2         2 

c'h     r      1        rp     (Xnj"XJ}      °j  ]  (5.46) 

c     Lfl  rL^^         4   Ei=l  4  2  w 


1        n  (9^/       1 

(tr  QE)'1(zP  .       J     I     )   L 

J  q  .0  • 


Combining   (5.18)   -   (5.19),    (5.27)   -   (5.29),    (5.37)  and   (5.46) 
for  m  >  6  and  9  t  x,  as  c  •*■  0, 


l/2                 l/2           trwV) 
R(e,Mj)   -  2c        (tr  QS)  +4 o  (5.47) 
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,       (0.-X.)    .   ? 

+  c  (tr  QE)_i(EP    J  I     )"i(b  -2b(p-2))  +  o(c)  . 
J     q  .a . 


For  9  =  X,  it  follows  from  (5.18)  that 


E       [ i : 

M  M2iXu-xi.j" 

~N   ~  W 


(5.48) 


f  r1  r00   p  t      _qJsNJ    _■ 

EEL¥  Jn    T  l~    4     7  -  2 
~         °  J-l  qJSNj  +  2taj 


v2 


dt]   . 


V2   a.s.  V2 


4  a«s.     . 
Since  Nc  +     (tr  QZ)         and  sN .     +     or,,   j=l,...,p,    then 


-v2  i 

c      i  r  *  (■ 


p  ,     qjsNj 


V2 

-)        dt 


0    .   .    q.s.. .  +  2ta. 
J=l  HJ  Nj  j 


(5.49) 


a.s. 


-y? 


(tr  QE)       "    /      JL{ 


q  .a. 
P   f__JJ 


-)  dt  . 


0    . \    q.a.  +  2t 
J=l     J   J 


Also, 


"  /2    i    r     »,, 

¥  J„     n    v       4 


P   ,       qJSNj 


v2 


r)  dt 


0    .",    q  .s7.   +  2ta' 
J=l   MJ  Nj  j 


(5.50) 


Vi 


<  K,(Nc       )    x     max     (s^.)p   , 
d  l<j<p       NJ 
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where  the  right  hand  side  of  (5.50)   is  uniformly  integrate  in 

c  <  c   ,   for  m  >  2   (K?  is  a  constant  independent  of  c).     Using   (5.48) 

-   (5.50)   one  gets  as  c  +  0,   for  m  >  2  and  9  =  X, 


-v2  , 

c  Efl    yt    j) T^ 


-  V2  q  -a2. 

♦   (tr  QE)       ~    /"    t  (— V )dt 

0   jil  Vd  +  2t 


Using  (5.28),  for  9  =  X,  we  have 


2 


(5.51) 


E     „[-*—* 5-eP  .  -4-]  (5.52) 

2*£  N2nX,-X,,2  J=1  s2 


,  E  ri  ,P  1L.  f    S.  i "h^M )dt] 

Nj    J-l  HJ  Nj      j 


Here,  as  c  +  0, 

-v?  ,      a2  q.sj.      y? 

a .  s .       -  V?       q  -o  ■  V? 

♦  p(tr  Qz)    T  ■?■(— V )   dt  . 

0  lx  ,/.   +  2t 


Further,  with  probability  one, 
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-V?    ,  o2  q.sj.  V2 

«  l'5-l-j-rn-f<     /     ]  2J         «  (5.54, 

s., .     0    .   .    q  .s.. .  +  2t0  . 
Nj  J=l   HJ   Nj  j 

<  K.   (Nc       )-1     max     (s~.2.)     max     (s.2,,)P   , 


4         l<j<P   Nj  l<j<P   Nj' 


where  the  right  hand  side  of  (5.54)  is  uniformly  integrable  in 
c  <  c  for  m  >  6  (K.  is  a  constant  independent  of  c).  Therefore, 
using  (5.52)  -  (5.54)  one  gets  as  c  *  0,  for  9  »  X  and  m  >  6, 


~N  ~  W       NJ 


'c.     ,<x>      n  .  ^i 


P(tr  QI)   "  f  r  (— V )dt 

0  '   q.a.  +  2t 
J=l  J  J 


Using   (5.39)   and   (5.42)  we   have   for  9   =  X, 


a2 


Efl  r[-?— S 5"  (  max    -9-):>  (5-56) 

~'~  N  "?N"~"W     1<-^V  SNj 


1  CT  •  H  -sm  •  ^2 

=  Ey[i-  (  max    -4-)  T      f  ( f-^ ?)    "    dt] 

1<J<P  sNj       0     j=1  qjSNj   +  2ta  . 


and 
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i      ,  _,  <\ 


E_  .[-7-i *•  (  min    -*-)]  (5.57) 

2  ~  N   "Vi'w     ^^  SNj 


a .  a  .su .  V? 


2  2 

a.  a.s.                          cr .      a.s. 

Here,   using  the  fact  that    min  (-4-)  ♦     1  and     max  (-^-)       ♦     1 

1<J<P  sNj  1<J<P  sNj 

as  c  ♦  0,  one  gets 


l/2    1  0i       ,»     n  qisNi  1/2 

1<J<P  S^       0    .=1  qjSNj  ♦  2ta  . 


|  (  max    _J_)  /      -P-  ( ^ r)     -    dt  (5.58) 


a.s.  -V2  q^.  V2 

♦  (tr  QE)  /      -?-  ( P )         dt 

~~  0    .   .    q.a.  +  2t 

J=l  HJ   J 


as  c  >  0,  and 


-  V?    ,  a  .  q  .sj  •  V? 

c      2  |  (  min    -f)  /"    f  (       4JNJ       2)         dt  (5.59) 

1<3<PS^        0  j=1  qjsjj  ♦  2ta  . 


a-S.  -V2  q,a.  V2 

♦   (tr  QE)  /      {•  ( p )         dt 

*"  0     '      q.a2:  +  2t 

J=l  MJ   J 


V2  -1 
Therefore,  using  the  uniform  integrabili ty  of  (Nc   )   max  — r— 

1<J<P  sNj 


144 


a.  Am  \ 


f     P  ( Jjy _)    "    dt  for  m  >  6  in  c  <  crt,  we  have  by   (5.38), 


(X,..-X.)2a2. 

c    ■  hit-T^—i1]--!-^ — r] 


(5.60) 


(trjffi)  J      f(     J2J     -)       dt 

0    . "      q.o.  +  2t 
J=l     J   J 


as  c  +  0  when  0  =  x  and  m  >  6.     Combining   (5.51),   (5.55)  and   (5.60) 
for  m  >  6  and  9  =  X,  as  c  ♦  0, 


V2  V2 


b  '2 

R(0,Z,«J)   =  2c        (tr   0>) 


(5.61) 


1/2  -  V2  q  -CT  •       V2 

+  c       (tr  01)         (b2-2b(p-2))  /"  r( H J       dt 

0  j=1  q^  ♦  2t 

+  o(cV2)    . 


From  (5.10),    (5.47)  and   (5.61)   it  follows  that 


R(0,£,XN)   -  R(e,E.*jj) 


(5.62) 


=  c(tr  QZ)_1{SP.     (0.-X   )2/(q  cr*)}"1  b(2(p-2)-b)   +  o(c) 

**"***  J ""  -i  J  J  J      J 


as  c  +  0  when  Q   t  X,   and 
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R(9,E,XN)  -  R(e,E,6Jj)  (5.63) 


c    b  (2(p-2)-b)  tr  (Qz)    /   f  ( f^ )   dt  +  o(c   ) 

0  j!1  ,/.   +  2t 


as  c  +  0  when  9  =  a.  The  leading  term  in  both  (5.62)  and  (5.63) 
is  maximized  when  b  =  p-2.  In  a  corresponding  fixed  sample  problem, 
there  is  only  some  numerical  evidence  in  Berger  and  Bock  (1976)  that 
b  =  p-1  may  be  an  optimal  choice.  The  discrepancy  between  our 
conclusions  needs  further  exploration. 
Remark 

As  of  this  time  we  are  unable  to  show  the  exact  dominance  of  s.. 
over  Xj..  However,  the  asymptotic  results  are  encouraging. 


CHAPTER  SIX 
SUMMARY  AND  FUTURE  RESEARCH 


6.1  Summary 

In  Chapter  2  we  considered  a  one  sample  problem  with  X.,  X2, 

2 

.    .    .    independent  and  identically  distributed  N(e,a  V)  where  0  and 

2 
a     were  unknown  and  V,  a  positive  definite  matrix,  was  known.     The 

problem  was  estimation  of  0.     The  loss  was  weighted  squared  error 

plus  cost  where  the  weight  matrix,  Q,  was  known  and  positive 

definite.     In  Chapter  3  we  considered  the  Gauss-Markoff  set-up  with 

Y     =  Z  0   +  e     where  Z     was  a  known  nxp  design  matrix,  g    (pxl)  was 

2  2 

unknown  and  e     was  distributed  N(0,a   I   )  where  a     was  unknown.     The 
~n  ~      ~n 

problem  was  estimation  of  6.     The  loss  was  again  weighted  squared 
error  plus  cost,   the  weight  matrix  being   (Z  Z   ).     In  Chapter  4  we 

considered  a  two  sample  problem  with  X..,   X„,    .    .    .   independent  and 

2 
identically  distributed  H(Q.to7V.)  and  Y.,  Yp,    .    .    .   independent  and 

2 

identically  distributed  N(02,a2V2),   independently  of  the   X. 's. 

2 
Here,  0.  and  cr.,  1=1,2  were  unknown  while  V.,   i=l,2,  was  a  known 

positive  definite  matrix.     The  problem  was  estimation  of  v  =  0,-0?. 

The  loss  again  was  weighted  squared  error  plus  cost  where   the  weight 

matrix,   Q,  was  known  and  positive  definite.      In  Chapter  5  we  returned 

to  the  one   sample  problem  with  X   ,   X   ,    .    .    .   independent  and 
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2  2 

identically  distributed  N(0,£)  where  9  and  £  =  diag(cr     . . .  ,cr  )  were 

unknown.     The  problem  was  estimation  of  £.     The   loss  again  was 

weighted  squared  error  plus  cost  where  the  weight  matrix  was 

Q  =  diag(q1,...,q  ),  q..  >  0,  j*l,...,p. 

In  all   cases  full   sequential   sampling  was  performed,  a  class  of 

estimators  was  shown   to  dominate  the  usual   estimator  (except  for 

Chapter  5),  and  an  asymptotic  risk  expansion  was  developed  for  the 

above  class  up  to  the  second  order  term. 

6.2     Future  Research 


First  we  should  return  to  the  situation  in  Chapter  5  and  try  to 
prove  the  exact  dominance  over  X.,  of  the  class  of  estimators  we 
considered.     Then  we  should  consider  the  above  problem  modified  to  a 
two  sample  situation  where  we  would  want  to  estimate   the  difference 
between  the  two  population  means.     Our  goal  will   be  to  develop  a 
class  of  estimators  dominating  the  usual   one  and  produce  an 
asymptotic  risk  expansion  for  the  above  class. 

Next  we  should  address  the  one   sample   situation  with  X,,   X?, 
.   .    .   independent  and  identically  distributed  N(Q,£)  where  £  e  Rp 
and  £,  positive  definite,  are  both  unknown.     The  problem  will   be 
estimation  of  9,   the  loss  would  be  weighted  squared  error  plus 
cost  where  the  weight  matrix,   Q,  would  be  positive  definite  and  the 
sampling  would  be  fully  sequential.     The  above  problem  has  been 
addressed  in  the  fixed  sample  case  by  Berger  and  Haff  (1983).     Then 
we  would  consider  the  above  problem  modified   to  a   two   sample 
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situation  where  we  would  want  to  estimate  the  difference  between  the 
two  population  means. 

Next,  we  should  return  to  all   the  problems  considered  in  this 
dissertation  and  the  ones  mentioned  above  and  do  three  additional 
things.     The  first  would  be  to  replace  the  constant(s)   in  the 
James-Stein  estimator  by  a  function  or  functions  of  the  observations 
which  would  be  bounded  between  0  and  2(p-2).     This  would  further 
broaden  the  class  of  estimators,  which  would  include   the  class  of 
constants,   possibly  allowing  greater  risk  reduction.     The  second 
would  be  to  replace  the  shrinkage  point(s)  by  the  grand  mean(s)   times 
a  vector  of  ones.     This  was  first  suggested  by  Lindley  (1962)  and 
would  perform  best  when   the  elements  of  the  mean  vector  are 
relatively  close  to  one  another.     Finally  we  would  like  to  consider 
two-stage  procedures.     The  reason  for  this  is  that  in  some  situations 
practical   constraints  may  allow  the  sampling  only  to  be  done  in  two 
stages  and  hence  a  full   sequential   sampling  scheme  would  not  be 
feasible. 
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